Collaborative Research: Elements: ProDM: Developing A Unified Progressive Data Management Library for Exascale Computational Science

协作研究：要素：ProDM：为百亿亿次计算科学开发统一的渐进式数据管理库

• 批准号: 2311757
• 负责人: Qing Liu
• 金额: $ 17.95万
• 依托单位: New Jersey Institute of Technology
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-08-01 至 2026-07-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2311757&HistoricalAwards=false
• 关键词: Collaborative; Research; Elements; ProDM; Developing

Effective management of scientific data produced by extreme-scale simulations and instruments is crucial for advancing scientific discoveries. Due to the scale of data and the diverse requirements of scientific analytics, there is a growing need to manage data in a progressive manner, such that users can stream as much data as they need to carry out their data analytics with reduced data movement and computation. However, little effort has been put into creating robust and scalable cyberinfrastructure services that link the recent algorithmic innovations in progressive methods with scientific data analytics, leaving these capabilities inaccessible to scientists. This project aims to develop a sustainable framework ProDM that supports the progressive management of scientific data to facilitate its use in scientific applications. The success of this project will enable new scientific research and novel findings by providing a new way to manage and analyze data. Furthermore, outcomes of this project will be delivered as publicly available software to enhance research cyberinfrastructure, promote education and teaching, and broaden participation in computing. ProDM is centered upon the unification of viable progressive representations and tailored development for in-situ and post-hoc analytic routines. In particular, it involves three key activities. First, a data engine will be built to unify state-of-the-art progressive representations, and provide portable hardware support for accelerators as well as interoperative software interfaces to other data management and analytic libraries. Second, an in-situ engine will be developed to facilitate the use of progressive representations for in-situ data analytics, which include a redesign of in-situ semantics and adjustment of runtime dynamics. Third, a post-hoc engine will be developed to efficiently access progressive data and improve the performance of data retrieval for post-hoc data analytics. ProDM will be deployed on campus-wide computing infrastructures and leadership systems for integration and evaluation with real-world scientific applications from climate, fusion, molecular dynamics, and beyond.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

有效管理由极端模拟和工具产生的科学数据对于推进科学发现至关重要。由于数据的规模和科学分析的各种要求，越来越需要以渐进的方式管理数据，以便用户可以通过减少数据移动和计算进行数据分析所需的尽可能多的数据流。但是，几乎没有努力创建强大而可扩展的网络基础架构服务，这些服务将渐进式方法中最近的算法创新与科学数据分析联系起来，从而使科学家无法访问这些功能。该项目旨在开发一种可持续的框架产品，以支持科学数据的逐步管理，以促进其在科学应用中的使用。该项目的成功将通过提供一种管理和分析数据的新方法来实现新的科学研究和新发现。此外，该项目的结果将作为公开软件提供，以增强研究网络基础设施，促进教育和教学以及扩大计算的参与。 PRODM以可行的渐进表示形式的统一和定制的现场分析程序的统一性为中心。特别是，它涉及三个关键活动。首先，将构建数据引擎以统一最新的渐进表示形式，并为加速器提供便携式硬件支持以及互操作软件接口到其他数据管理和分析库。其次，将开发一个原位引擎，以促进渐进表示形式用于原位数据分析，其中包括对原位语义的重新设计和调整运行时动力学的调整。第三，将开发事后引擎，以有效地访问渐进数据并改善事后数据分析的数据检索的性能。产品将部署在校园范围内的计算基础架构和领导系统上，以与气候，融合，分子动力学以及其他超越现实世界的科学应用进行整合和评估。该奖项反映了NSF的法定任务，并已通过该基金会的智力功能和广泛的影响来评估NSF的法定任务，并被认为是值得的。