Collaborative Research: Elements: ROCCI: Integrated Cyberinfrastructure for In Situ Lossy Compression Optimization Based on Post Hoc Analysis Requirements

合作研究：要素：ROCCI：基于事后分析要求的原位有损压缩优化的集成网络基础设施

• 批准号: 2247080
• 负责人: Dingwen Tao
• 金额: $ 28万
• 依托单位: Indiana University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-10-01 至 2024-10-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2247080&HistoricalAwards=false
• 关键词: Collaborative; Research; Elements; ROCCI; Integrated

Today’s simulations and advanced instruments are producing vast volumes of data, presenting a major storage and I/O burden for scientists. Error-bounded lossy compressors, which can significantly reduce the data volume while controlling data distortion with a constant error bound, have been developed for years. However, a significant gap still remains in practice. On the one hand, the impact of the compression errors on scientific research is not well understood, so how to set an appropriate error bound for lossy compression is very challenging. On the other hand, how to select the best fit compression technology and run it automatically in scientific application codes is non-trivial because of strengths and weaknesses of different compression techniques and diverse characteristics of applications and datasets. This project aims to develop a Requirement-Oriented Compression Cyber-Infrastructure (ROCCI) for data-intensive domains such as astrophysics and materials science, which can select and run the best fit lossy compressor automatically at runtime, in terms of user's requirement on their post hoc analysis.The overarching goal of this project is to offer a complete series of automatic functions and services allowing users to transparently run the best fit compressor at runtime during the scientific simulations or data acquisition. This project advances knowledge and understanding with three key thrusts: (1) it builds an efficient layer to interoperate with different lossy compressors and diverse post hoc analysis requirements on data fidelity by leveraging an existing compression adaptor library (LibPressio) and compression assessment library (Z-checker); (2) it develops an efficient engine to determine the best fit compressor with optimized settings based on user’s post-hoc analysis requirements; and (3) it develops a user-friendly infrastructure that integrates compression optimization and execution via the HDF5 dynamic filter mechanism. This project particularly targets cosmology and materials science applications and their specific requirements of using lossy compressors in practice.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

当今的仿真和高级仪器正在生产大量数据，并为科学家提供了重大存储，并燃烧了I/O。多年来，已经开发了误差损耗压缩机，该压缩机可以显着减少数据量，同时以恒定的误差绑定来控制数据量失真。但是，实践中仍然存在很大的差距。一方面，尚不清楚压缩错误对科学研究的影响，因此如何为损失压缩设定适当的错误是非常挑战的。另一方面，如何选择最佳拟合压缩技术并在科学应用程序代码中自动运行它，这是非平凡的，因为不同压缩技术的优势和劣势以及应用程序和数据集的各种特征。该项目旨在为数据密集型领域（例如天体物理学和材料科学）开发面向需求的压缩网络内部结构（ROCCI），可以选择和运行最佳合适的有损压缩机，在运行时自动在运行时，就用户在HOC的速度分析方面，该项目允许自动跑步的跑步型和跑步的跑步。科学模拟或数据获取。该项目通过三个关键的推动力提高知识和理解：（1）它通过利用现有的压缩适配器库（LIBPRESSIO）（libpressio）和压缩评估库（z-Checkecker）（Z-Checker）（Z-Checker）来建立与不同的损耗压缩机相互互操作的有效层和有关数据保真度的不同之后的层； （2）根据用户的事后分析要求，它开发了有效的引擎，以确定最佳拟合压缩机，并通过优化的设置； （3）它开发了一种用户友好的基础架构，该基础架构通过HDF5动态滤波器机制集成了压缩优化和执行。该项目尤其针对宇宙学和材料科学应用及其在实践中使用有损压缩机的具体要求。该奖项反映了NSF的法定任务，并且使用基金会的知识分子优点和更广泛的影响审查标准，被视为通过评估来获得珍贵的支持。

项目成果

cuZ-Checker: A GPU-Based Ultra-Fast Assessment System for Lossy Compressions

cuZ-Checker：基于 GPU 的有损压缩超快速评估系统

• DOI: 10.1109/cluster48925.2021.00065
• 发表时间: 2021
• 期刊: 2021 IEEE International Conference on Cluster Computing (CLUSTER 2021
• 作者: Yu, Xiaodong;Di, Sheng;Gok, Ali Murat;Tao, Dingwen;Cappello, Franck
• 通讯作者: Cappello, Franck

Ultrafast Error-Bounded Lossy Compression for Scientific Datasets

科学数据集的超快误差限制有损压缩

• DOI: 10.1145/3502181.3531473
• 发表时间: 2022
• 期刊: The 31st ACM International Symposium on High-Performance Parallel and Distributed Computing (HPDC 2022
• 作者: Yu, Xiaodong;Di, Sheng;Zhao, Kai;Tian, Jiannan;Tao, Dingwen;Liang, Xin;Cappello, Franck
• 通讯作者: Cappello, Franck

Optimizing Error-Bounded Lossy Compression for Scientific Data With Diverse Constraints

优化具有不同约束的科学数据的误差有限有损压缩

• DOI: 10.1109/tpds.2022.3194695
• 发表时间: 2022
• 期刊: IEEE Transactions on Parallel and Distributed Systems
• 影响因子: 5.3
• 作者: Liu, Yuanjian;Di, Sheng;Zhao, Kai;Jin, Sian;Wang, Cheng;Chard, Kyle;Tao, Dingwen;Foster, Ian;Cappello, Franck
• 通讯作者: Cappello, Franck

Optimizing Error-Bounded Lossy Compression for Scientific Data on GPUs

优化 GPU 上科学数据的误差有限有损压缩

• DOI: 10.1109/cluster48925.2021.00047
• 发表时间: 2021
• 期刊: 2021 IEEE International Conference on Cluster Computing (CLUSTER 2021
• 作者: Tian, Jiannan;Di, Sheng;Yu, Xiaodong;Rivera, Cody;Zhao, Kai;Jin, Sian;Feng, Yunhe;Liang, Xin;Tao, Dingwen;Cappello, Franck
• 通讯作者: Cappello, Franck

FZ-GPU: A Fast and High-Ratio Lossy Compressor for Scientific Computing Applications on GPUs

• DOI: 10.1145/3588195.3592994
• 发表时间: 2023-04
• 期刊: Proceedings of the 32nd International Symposium on High-Performance Parallel and Distributed Computing
• 作者: Bo Zhang;Jiannan Tian;S. Di;Xiaodong Yu;Yunhe Feng;Xin Liang;Dingwen Tao;F. Cappello