SDRBench: Scientific Data Reduction Benchmark for Lossy Compressors

Efficient error-controlled lossy compressors are becoming critical to the success of today’s large-scale scientific applications because of the ever-increasing volume of data produced by the applications. In the past decade, many lossless and lossy compressors have been developed with distinct design principles for different scientific datasets in largely diverse scientific domains. In order to support researchers and users assessing and comparing compressors in a fair and convenient way, we establish a standard compression assessment benchmark – Scientific Data Reduction Benchmark (SDRBench)1. SDRBench contains a vast variety of real-world scientific datasets across different domains, summarizes several critical compression quality evaluation metrics, and integrates many state-of-the-art lossy and lossless compressors. We demonstrate evaluation results using SDRBench and summarize six valuable takeaways that are helpful to the in-depth understanding of lossy compressors.

由于大规模科学应用产生的数据量不断增加，高效的误差控制有损压缩器正成为这些应用取得成功的关键。在过去十年中，针对不同科学领域的不同科学数据集，开发了许多具有不同设计原则的无损和有损压缩器。为了支持研究人员和用户以公平、便捷的方式评估和比较压缩器，我们建立了一个标准压缩评估基准--科学数据压缩基准（SDRBench）1。SDRBench 包含大量不同领域的真实科学数据集，总结了几个关键的压缩质量评估指标，并集成了许多最先进的有损和无损压缩器。我们利用 SDRBench 展示了评估结果，并总结了有助于深入了解有损压缩器的六条宝贵经验。