CAREER: Enabling Progressive Data Analytics for High Performance Computing: Algorithms and System Support

职业：实现高性能计算的渐进式数据分析：算法和系统支持

• 批准号: 2144403
• 负责人: Qing Liu
• 金额: $ 49.97万
• 依托单位: New Jersey Institute of Technology
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-04-01 至 2027-03-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2144403&HistoricalAwards=false
• 关键词: CAREER; Enabling; Progressive; Data; Analytics

This award is funded in whole or in part under the American Rescue Plan Act of 2021 (Public Law 117-2).Rapidly extracting new knowledge from simulation output is critical to the computational sciences at high performance computing (HPC) facilities across the country. However, this has become increasingly challenging due to the growing disparity between the volume of data produced by simulations and the ability to post process the data at the rate it is produced. This project aims to explore reduced representations of data with the overarching goal of achieving science aware and highly adaptable data analytics for HPC applications. The project will create new algorithms and software systems, and benefit the current and future cyberinfrastructure in the U.S. as well as numerous data intensive scientific applications, such as nuclear fusion, astrophysics, combustion, earth science, and others, thus reinforcing the competitiveness and leadership of the United States in this area. Success in the project goals will greatly reduce the time to new knowledge from scientific simulations across various science and engineering disciplines at HPC centers and significantly enhance HPC research and education. The project will contribute to society through engaging underrepresented groups and a set of integrated research and education activities.The project will develop algorithms and system support centered on the idea of leveraging multilevel data representations to enable progressive data analytics on HPC systems. The proposed work fundamentally differs from conventional lossy data compression in that it can guarantee and enforce scientific constraints and augment accuracy based upon applications needs and system state. The project has integrated research and educational activities in algorithms, systems, and applications, taking into account application requirements and architecture trends in large-scale storage to advance the field of scientific data management. More specifically, the project will make contributions in several areas: 1) constraint-based data decomposition; 2) exploiting error-controlled multilevel representations for performance optimization on HPC storage systems; 3) providing a cross-layer solution to mitigate performance variation in containerized environments, with multiprocessor and multi-application coordination achieved through a probabilistic method for selecting the number of levels to retrieve; and 4) integration and evaluation on production science applications.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该奖项是根据2021年《美国救援计划法》的全部或部分资助的（公共法117-2）。从模拟输出中逐渐提取新知识对于全国高性能计算（HPC）设施的计算科学至关重要。但是，由于模拟产生的数据量与以其生成的速率处理数据的能力之间的差异日益增加，因此这变得越来越具有挑战性。该项目旨在探索数据的减少表示形式，即为HPC应用程序实现科学意识和高度适应性数据分析的总体目标。该项目将创建新的算法和软件系统，并使美国当前和未来的网络基础设施受益，以及许多数据密集的科学应用，例如核融合，天体物理学，燃烧，地球科学等，从而增强了竞争力和领导力在该地区的美国。项目目标的成功将大大减少来自HPC中心各种科学和工程学科的科学模拟的新知识的时间，并显着增强HPC研究和教育。该项目将通过使代表性不足的群体和一组综合的研究和教育活动为社会做出贡献。该项目将开发算法和系统支持以利用多级数据表示形式的想法，以启用HPC系统的渐进数据分析。拟议的工作从根本上与常规的损耗数据压缩有所不同，因为它可以根据应用需求和系统状态来保证和实施科学限制和增强准确性。该项目考虑了算法，系统和应用程序中的研究和教育活动，并考虑了大规模存储的应用要求和建筑趋势，以推动科学数据管理领域。更具体地说，该项目将在多个领域做出贡献：1）基于约束的数据分解； 2）利用错误控制的多级表示，以在HPC存储系统上进行性能优化； 3）提供一种跨层解决方案来减轻容器化环境中的性能变化，多处理器和多应用协调通过一种概率方法来选择要检索的级别的数量； 4）对生产科学应用的整合和评估。该奖项反映了NSF的法定任务，并被认为是使用基金会的知识分子优点和更广泛影响的评论标准的评估值得支持的。

项目成果

RAPIDS: Reconciling Availability, Accuracy, and Performance in Managing Geo-Distributed Scientific Data

RAPIDS：协调管理地理分布式科学数据的可用性、准确性和性能

• DOI: 10.1145/3588195.3592983
• 发表时间: 2023
• 期刊: ACM
• 作者: Wan, Lipeng;Chen, Jieyang;Liang, Xin;Gainaru, Ana;Gong, Qian;Liu, Qing;Whitney, Ben;Arulraj, Joy;Liu, Zhengchun;Foster, Ian
• 通讯作者: Foster, Ian

MGARD: A multigrid framework for high-performance, error-controlled data compression and refactoring

MGARD：用于高性能、错误控制数据压缩和重构的多重网格框架

• DOI: 10.1016/j.softx.2023.101590
• 发表时间: 2023
• 期刊: SoftwareX
• 影响因子: 3.4
• 作者: Gong, Qian;Chen, Jieyang;Whitney, Ben;Liang, Xin;Reshniak, Viktor;Banerjee, Tania;Lee, Jaemoon;Rangarajan, Anand;Wan, Lipeng;Vidal, Nicolas
• 通讯作者: Vidal, Nicolas

Zperf: A Statistical Gray-Box Approach to Performance Modeling and Extrapolation for Scientific Lossy Compression

Zperf：科学有损压缩性能建模和外推的统计灰盒方法

• DOI: 10.1109/tc.2023.3257517
• 发表时间: 2023
• 期刊: IEEE Transactions on Computers
• 影响因子: 3.7
• 作者: Wang, Jinzhen;Chen, Qi;Liu, Tong;Liu, Qing;He, Xubin
• 通讯作者: He, Xubin

Improving Progressive Retrieval for HPC Scientific Data using Deep Neural Network

使用深度神经网络改进 HPC 科学数据的渐进检索

• DOI: 10.1109/icde55515.2023.00209
• 发表时间: 2023
• 期刊: IEEE
• 作者: Wang, Jinzhen;Liang, Xin;Whitney, Ben;Chen, Jieyang;Gong, Qian;He, Xubin;Wan, Lipeng;Klasky, Scott;Podhorszki, Norbert;Liu, Qing
• 通讯作者: Liu, Qing