CC*IIE Integration: Collaborative Research: EPSON: Embracing Parallel Networks and Storage for Predictable End-to-End Data Movement

CC*IIE 集成：协作研究：EPSON：采用并行网络和存储实现可预测的端到端数据移动

• 批准号: 1440797
• 负责人: Michael Papka
• 金额: $ 27.99万
• 依托单位: Northern Illinois University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2014
• 资助国家: 美国
• 起止时间: 2014-10-01 至 2017-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1440797&HistoricalAwards=false
• 关键词: CC; IIE; Integration; Collaborative; Research

Geographically distributed scientific communities require increasingly sophisticated data transfer mechanisms that can handle the challenges of sharing large datasets over heterogeneous networks. These challenges include optimization of networks with different configurations and protocols, I/O mechanisms to efficiently read and write to parallel storage, and the varying demands of widely different data transfer workloads.To address these challenges, the EPSON project is developing, implementing, and evaluating application programming interfaces and tools that facilitate end-to-end parallel data transfers. EPSON researchers focus on three areas: (1) enabling parallel network data movement, by taking into account the diversity of parallel network characteristics of both shared networks and infrastructures with dedicated circuits and paths and effectively balancing the flows among paths for more predictable performance; (2) developing a GridFTP data storage interface, enabling scalable I/O to and from parallel filesystems -- critical for campus infrastructures to deal with large-scale datasets; and (3) devising mechanisms that overlap network transfers with storage I/O and incorporate data-staging heuristics, matching the impedance between storage and networking capabilities to improve end-to-end data transfers. The project involves close collaboration with application scientists, with the objective of providing advanced networking tools to support the requirements of the applications deployed at the University of Chicago and Northern Illinois University campuses.

地理上分布的科学界需要日益复杂的数据传输机制，以应对通过异构网络共享大型数据集的挑战。这些挑战包括优化具有不同配置和协议的网络、有效读写并行存储的 I/O 机制，以及不同数据传输工作负载的不同需求。为了应对这些挑战，EPSON 项目正在开发、实施和实施评估促进端到端并行数据传输的应用程序编程接口和工具。 EPSON研究人员重点关注三个领域：（1）通过考虑共享网络和具有专用电路和路径的基础设施的并行网络特征的多样性，实现并行网络数据移动，并有效平衡路径之间的流量以获得更可预测的性能； (2) 开发 GridFTP 数据存储接口，支持并行文件系统之间的可扩展 I/O——这对于校园基础设施处理大规模数据集至关重要； (3) 设计将网络传输与存储 I/O 重叠的机制，并结合数据分级启发法，匹配存储和网络功能之间的阻抗，以改善端到端数据传输。该项目涉及与应用科学家的密切合作，目的是提供先进的网络工具来支持芝加哥大学和北伊利诺伊大学校园部署的应用程序的要求。