CDS&E: Collaborative Research: HyLoC: Objective-driven Adaptive Hybrid Lossy Compression Framework for Extreme-Scale Scientific Applications

CDS

• 批准号: 2303064
• 负责人: Dingwen Tao
• 金额: $ 27.08万
• 依托单位: Indiana University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-10-01 至 2024-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2303064&HistoricalAwards=false
• 关键词: CDS& Collaborative; Research; HyLoC; Objective

Today's extreme-scale scientific simulations and instruments are producing huge amounts of data that cannot be transmitted or stored effectively. Lossy compression, a data compression approach leading to certain data distortion, has been considered as a promising solution, because it can significantly reduce the data size while maintaining high data fidelity. However, the existing lossy compression methods may not always work effectively on all datasets used in specific applications because of their distinct and diverse characteristics. Moreover, the user objectives in compression quality and performance may vary with applications, datasets or circumstances. This project aims to develop a hybrid lossy compression framework to automatically construct the best-fit compression for diverse user objectives in data-intensive scientific research. Educational and engagement activities are provided to develop new curriculum related to scientific data compression and promote research collaborations with national laboratories.Designing an efficient, adaptive, hybrid framework that can always choose the best-fit compression strategy is nontrivial, since existing state-of-the-art lossy compression methods are developed with distinct principles. The project has a three-stage research plan. First, the project decouples the state-of-the-art error-bounded lossy compression approaches into multiple stages and effectively models the working efficiency (e.g., compression ratio, error, speed) of particular approaches in each stage. Second, the project develops a loosely-coupled framework to aggregate the decoupled compression stages together and also explores as many compression pipelines composed of different stages as possible, to optimize the classic compression efficiency, including compression quality and performance. Third, the project optimizes the synthetic data-movement performance regarding the external devices and resources, such as I/O performance. The team evaluates the proposed framework on multiple extreme-scale scientific applications, including cosmological simulations, light source instrument data analytics, quantum circuit simulations, and climate simulations. The project may create technologies that can increase the storage availability and improve the performance for extreme-scale scientific applications, opening opportunities for new discoveries.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

当今的极端科学模拟和仪器正在生产大量无法传输或有效存储的数据。有损耗的压缩是导致某些数据失真的数据压缩方法，被认为是一个有前途的解决方案，因为它可以显着减少数据大小，同时保持高数据保真度。但是，由于其独特和多样化的特征，现有的损耗压缩方法可能并不总是在特定应用程序中使用的所有数据集有效地工作。此外，压缩质量和性能方面的用户目标可能随应用程序，数据集或情况而变化。该项目旨在开发一个混合有损压缩框架，以自动为数据密集型科学研究中的各种用户目标构建最佳合适的压缩。提供了教育和参与活动，以开发与科学数据压缩有关的新课程，并促进与国家实验室的研究合作。设计一种高效，适应性，混合的框架，该框架始终可以选择最佳拟合压缩策略，因为现有的现有最新的最先进的损失压缩方法具有独特的原理开发。该项目有三阶段的研究计划。首先，该项目将最新的错误结合的有损压缩方法分解为多个阶段，并有效地模拟了每个阶段特定方法的工作效率（例如，压缩比，误差，速度，速度）。其次，该项目开发了一个松散耦合的框架，以将解耦压缩阶段汇总在一起，并探索尽可能多的压缩管道，这些压缩管道由不同阶段组成，以优化经典的压缩效率，包括压缩质量和性能。第三，该项目优化了有关外部设备和资源（例如I/O性能）的合成数据移动性能。该团队在多个极端的科学应用上评估了所提出的框架，包括宇宙学模拟，光源仪器数据分析，量子电路模拟和气候模拟。该项目可能会创建可以提高存储可用性并提高极端科学应用的绩效的技术，这是新发现的开放机会。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子的评估和更广泛的影响来获得支持的。

项目成果

TSM2X: High-performance tall-and-skinny matrix-matrix multiplication on GPUs

• DOI: 10.1016/j.jpdc.2021.02.013
• 发表时间: 2020-02
• 期刊: J. Parallel Distributed Comput.
• 作者: Cody Rivera;Jieyang Chen;Nan Xiong;Jing Zhang;S. Song;Dingwen Tao

Ultrafast Error-Bounded Lossy Compression for Scientific Datasets

科学数据集的超快误差限制有损压缩

• DOI: 10.1145/3502181.3531473
• 发表时间: 2022
• 期刊: The 31st ACM International Symposium on High-Performance Parallel and Distributed Computing (HPDC 2022
• 作者: Yu, Xiaodong;Di, Sheng;Zhao, Kai;Tian, Jiannan;Tao, Dingwen;Liang, Xin;Cappello, Franck
• 通讯作者: Cappello, Franck

Optimizing Error-Bounded Lossy Compression for Scientific Data With Diverse Constraints

优化具有不同约束的科学数据的误差有限有损压缩

• DOI: 10.1109/tpds.2022.3194695
• 发表时间: 2022
• 期刊: IEEE Transactions on Parallel and Distributed Systems
• 影响因子: 5.3
• 作者: Liu, Yuanjian;Di, Sheng;Zhao, Kai;Jin, Sian;Wang, Cheng;Chard, Kyle;Tao, Dingwen;Foster, Ian;Cappello, Franck
• 通讯作者: Cappello, Franck

Optimizing Error-Bounded Lossy Compression for Scientific Data on GPUs

优化 GPU 上科学数据的误差有限有损压缩

• DOI: 10.1109/cluster48925.2021.00047
• 发表时间: 2021
• 期刊: 2021 IEEE International Conference on Cluster Computing (CLUSTER 2021
• 作者: Tian, Jiannan;Di, Sheng;Yu, Xiaodong;Rivera, Cody;Zhao, Kai;Jin, Sian;Feng, Yunhe;Liang, Xin;Tao, Dingwen;Cappello, Franck
• 通讯作者: Cappello, Franck

FZ-GPU: A Fast and High-Ratio Lossy Compressor for Scientific Computing Applications on GPUs

• DOI: 10.1145/3588195.3592994
• 发表时间: 2023-04
• 期刊: Proceedings of the 32nd International Symposium on High-Performance Parallel and Distributed Computing
• 作者: Bo Zhang;Jiannan Tian;S. Di;Xiaodong Yu;Yunhe Feng;Xin Liang;Dingwen Tao;F. Cappello