Collaborative Research: Application-adaptive I/O Stack for Data-intensive Scientific Computing

协作研究：用于数据密集型科学计算的应用自适应 I/O 堆栈

• 批准号: 0621427
• 负责人: Anand Sivasubramaniam
• 金额: $ 2万
• 依托单位: Pennsylvania State Univ University Park
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2006
• 资助国家: 美国
• 起止时间: 2006-09-15 至 2009-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=0621427&HistoricalAwards=false
• 关键词: Collaborative; Research; Application; adaptive; Stack

Advances in computational sciences have been greatly accelerated by the rapid growth of high-end computing (HEC) facilities. However, the continuous speedup of end-to-end scientific discovery cycles relies on the ability to store, share, and analyze the terabytes and petabytes of data generated by today's supercomputers. With the growing performance gap between I/O systems and processor/memory units, data storage and accesses are inevitably becoming more bottleneck-prone. In this proposal, we address the I/O stack performance problem with adaptive optimizations at multiple layers of the HEC I/O stack (from high-level scientific data libraries to secondary storage devices and archiving systems), and propose effective communication schemes to integrate such optimizations across layers. In particular, our proposed PATIO (Parallel AdapTive I/O) framework explores multi-layer caching/prefetching that coordinates storage resources ranging from processors to tape archiving systems. This novel approach will bridge existing disjoint optimization efforts at each individual layer and responds to the critical call of improving the overall I/O system performance with increasingly deep HEC I/O stacks.

高端计算 (HEC) 设施的快速增长极大地加速了计算科学的进步。然而，端到端科学发现周期的持续加速依赖于存储、共享和分析当今超级计算机生成的 TB 和 PB 数据的能力。随着 I/O 系统和处理器/内存单元之间的性能差距不断扩大，数据存储和访问不可避免地变得更容易出现瓶颈。在本提案中，我们通过 HEC I/O 堆栈多层（从高级科学数据库到辅助存储设备和归档系统）的自适应优化来解决 I/O 堆栈性能问题，并提出有效的通信方案来集成此类跨层优化。特别是，我们提出的 PATIO（并行自适应 I/O）框架探索了多层缓存/预取，协调从处理器到磁带归档系统的存储资源。这种新颖的方法将弥合每个单独层现有的不相交优化工作，并响应通过越来越深的 HEC I/O 堆栈提高整体 I/O 系统性能的关键要求。