Collaborative Research: Frameworks: FZ: A fine-tunable cyberinfrastructure framework to streamline specialized lossy compression development

合作研究：框架：FZ：一个可微调的网络基础设施框架，用于简化专门的有损压缩开发

• 批准号: 2311876
• 负责人: Dingwen Tao
• 金额: $ 58万
• 依托单位: Indiana University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-08-01 至 2027-07-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2311876&HistoricalAwards=false
• 关键词: Collaborative; Research; Frameworks; FZ; fine

Data is the fourth pillar of the science methodology. However, rapidly expanding volumes and velocities of scientific data generated by simulation and instrument facilities present serious storage capacity, storage and network bandwidth, and data analysis challenges for many sciences. These challenges ultimately limit research discovery which would promote prosperity and welfare. Many research groups are exploring the use of data reduction techniques to address these challenges because lossy compression for scientific data offers a reliable, high-speed, and high-fidelity solution. However, existing generic lossy compressors often do not correspond to user-specific applications, use cases, and requirements in terms of reduction, speed, and information preservation. Hence, many potential users of lossy compressors for scientific data develop their own specialized lossy compression software, an effort that requires tremendous collaboration between compressor experts and domain scientists, demands extensive coding to optimize performance on multiple platforms, and often leads to redundant research and development efforts. This project aims to create a framework, called FZ, that revolutionizes the development of specialized lossy compressors by providing a comprehensive ecosystem to enable scientific users to intuitively research, compose, implement, and test specialized lossy compressors from a library of pre-developed, high-performance data reduction modules optimized for heterogeneous platforms. This project also contributes to the education and training of undergraduate and graduate students by enhancing the quality of computing-related curricula in scientific data management, compression, and visualization and through outreach activities at four universities. This project builds FZ, an intuitive cyberinfrastructure for the composition of specialized lossy compressors, by adapting, combining, and extending multiple existing capabilities from the SZ lossy compressor, the LibPressio unifying compression interface, the OptZConfig optimizer of compressor configurations, the Z-checker and QCAT compression quality analysis tools, and the Paraview and VTK visualization tools. The project has three thrusts: (1) It develops programming interfaces and a compressor generator to create new compressors from high-level languages such as Python and optimize their execution. (2) It refactors the SZ lossy compressors infrastructure to enable fine-grained composability of a large diversity of data transformation modules and integrate non-uniform compression capabilities, new preprocessing, decorrelation, approximation, and entropy coding data transformation modules to produce specialized lossy compressors. (3) It provides interactive visualization, quality assessment, and graphical user interface (GUI) tools that adapt and extend existing capabilities to automatically search optimized lossy compression module compositions and to identify relevant compression ratio, speed, and quality trade-offs for their use cases.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

数据是科学方法论的第四个支柱。然而，模拟和仪器设施生成的科学数据的数量和速度迅速增长，给许多科学领域带来了严峻的存储容量、存储和网络带宽以及数据分析挑战。这些挑战最终限制了促进繁荣和福利的研究发现。许多研究小组正在探索使用数据缩减技术来应对这些挑战，因为科学数据的有损压缩提供了可靠、高速和高保真的解决方案。然而，现有的通用有损压缩器通常不符合用户特定的应用、用例以及在缩减、速度和信息保存方面的要求。因此，科学数据有损压缩器的许多潜在用户开发了自己的专用有损压缩软件，这项工作需要压缩器专家和领域科学家之间的大量协作，需要大量编码以优化多个平台上的性能，并且常常导致冗余的研发努力。该项目旨在创建一个名为 FZ 的框架，通过提供一个全面的生态系统，使科学用户能够从预先开发的高可用库中直观地研究、编写、实施和测试专用有损压缩机，从而彻底改变专用有损压缩机的开发。 - 针对异构平台优化的性能数据缩减模块。该项目还通过提高科学数据管理、压缩和可视化方面的计算相关课程的质量以及通过四所大学的推广活动，为本科生和研究生的教育和培训做出贡献。该项目通过调整、组合和扩展 SZ 有损压缩机、LibPressio 统一压缩接口、压缩机配置的 OptZConfig 优化器、Z-checker 和QCAT 压缩质量分析工具，以及 Paraview 和 VTK 可视化工具。该项目有三个主旨：（1）开发编程接口和压缩器生成器，以从Python等高级语言创建新的压缩器并优化其执行。 (2)它重构了SZ有损压缩器基础设施，以实现多种数据转换模块的细粒度可组合性，并集成非均匀压缩功能、新的预处理、去相关、近似和熵编码数据转换模块，以产生专门的有损压缩器。 (3) 它提供交互式可视化、质量评估和图形用户界面 (GUI) 工具，这些工具适应和扩展现有功能，以自动搜索优化的有损压缩模块组合，并确定其使用的相关压缩比、速度和质量权衡该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。