Collaborative Research: Frameworks: FZ: A fine-tunable cyberinfrastructure framework to streamline specialized lossy compression development

合作研究：框架：FZ：一个可微调的网络基础设施框架，用于简化专门的有损压缩开发

• 批准号: 2311878
• 负责人: Hanqi Guo
• 金额: $ 57.99万
• 依托单位: Ohio State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-08-01 至 2027-07-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2311878&HistoricalAwards=false
• 关键词: Collaborative; Research; Frameworks; FZ; fine

Data is the fourth pillar of the science methodology. However, rapidly expanding volumes and velocities of scientific data generated by simulation and instrument facilities present serious storage capacity, storage and network bandwidth, and data analysis challenges for many sciences. These challenges ultimately limit research discovery which would promote prosperity and welfare. Many research groups are exploring the use of data reduction techniques to address these challenges because lossy compression for scientific data offers a reliable, high-speed, and high-fidelity solution. However, existing generic lossy compressors often do not correspond to user-specific applications, use cases, and requirements in terms of reduction, speed, and information preservation. Hence, many potential users of lossy compressors for scientific data develop their own specialized lossy compression software, an effort that requires tremendous collaboration between compressor experts and domain scientists, demands extensive coding to optimize performance on multiple platforms, and often leads to redundant research and development efforts. This project aims to create a framework, called FZ, that revolutionizes the development of specialized lossy compressors by providing a comprehensive ecosystem to enable scientific users to intuitively research, compose, implement, and test specialized lossy compressors from a library of pre-developed, high-performance data reduction modules optimized for heterogeneous platforms. This project also contributes to the education and training of undergraduate and graduate students by enhancing the quality of computing-related curricula in scientific data management, compression, and visualization and through outreach activities at four universities. This project builds FZ, an intuitive cyberinfrastructure for the composition of specialized lossy compressors, by adapting, combining, and extending multiple existing capabilities from the SZ lossy compressor, the LibPressio unifying compression interface, the OptZConfig optimizer of compressor configurations, the Z-checker and QCAT compression quality analysis tools, and the Paraview and VTK visualization tools. The project has three thrusts: (1) It develops programming interfaces and a compressor generator to create new compressors from high-level languages such as Python and optimize their execution. (2) It refactors the SZ lossy compressors infrastructure to enable fine-grained composability of a large diversity of data transformation modules and integrate non-uniform compression capabilities, new preprocessing, decorrelation, approximation, and entropy coding data transformation modules to produce specialized lossy compressors. (3) It provides interactive visualization, quality assessment, and graphical user interface (GUI) tools that adapt and extend existing capabilities to automatically search optimized lossy compression module compositions and to identify relevant compression ratio, speed, and quality trade-offs for their use cases.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

数据是科学方法论的第四个支柱。但是，通过模拟和仪器设施生成的科学数据的量和速度迅速扩大，呈现出严重的存储容量，存储和网络带宽以及对许多科学的数据分析挑战。这些挑战最终会限制研究发现，这将促进繁荣和福利。许多研究小组正在探索使用数据减少技术来应对这些挑战的使用，因为科学数据的损失压缩提供了可靠，高速和高保真的解决方案。但是，现有的通用损失压缩机通常与降低，速度和信息保存方面的用户特定应用程序，用例和要求不符。因此，许多用于科学数据的有损压缩机的潜在使用者开发了自己的专业有损压缩软件，这项工作需要压缩机专家与域科学家之间进行巨大的合作，需要大量编码以优化多个平台上的性能，并且通常会导致冗余的研究和开发工作。该项目旨在创建一个称为FZ的框架，该框架通过提供一个全面的生态系统来彻底改变专门的有损压缩机的开发，以使科学用户能够从预先开发的，高性能数据减少的模块优化异质平台的库中直观地研究，组成，实施和测试专业的有损压缩机。该项目还通过提高科学数据管理，压缩和可视化与计算相关课程的质量以及通过四所大学的外展活动来提高与计算相关课程的质量，从而有助于对本科和研究生的教育和培训。该项目通过适应，合并和扩展SZ损耗压缩机的多个现有功能，libpressio统一压缩界面，OptzConconfig优化的压缩机配置的OPTZCONCONFIG优化器，Z-Checkecter和QCAT级别的工具和QCAT级别的工具和QCAT质量和QCAT质量质量，从而构建了FZ，这是一种直观的网络基础结构，可用于专业有损压缩机的组成。该项目具有三个推力：（1）它开发了编程接口和一个压缩机生成器，以从Python等高级语言中创建新的压缩机，并优化其执行。 （2）它重构SZ损耗的压缩机基础架构，以实现大量数据转换模块的精细元素合成性，并整合了非均匀的压缩能力，新的预处理，去相关，近似，近似值和熵编码编码模块，以产生专业的压缩损失损失的压缩损失。 （3）它提供了交互式可视化，质量评估和图形用户界面（GUI）工具，可适应和扩展现有功能，以自动搜索优化的有损压缩模块组成，并确定相关的压缩比率，速度和优质折衷的用例，以反映NSF的法定任务，并通过评估构成的范围来表现出色的范围，以表现出众多的依据。