Collaborative Research: CNS Core: Small: A New Architecture for Petabyte-scale File Transfer Evaluated in FABRIC

合作研究：CNS 核心：小型：在 FABRIC 中评估的 PB 级文件传输新架构

• 批准号: 2215671
• 负责人: Violet Syrotiuk
• 金额: $ 32.4万
• 依托单位: Arizona State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-10-01 至 2025-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2215671&HistoricalAwards=false
• 关键词: Collaborative; Research; CNS; Core; Small

File transfer is a fundamental operation of the Internet. Important scientific instruments, such as the James Webb Space Telescope and the Large Hadron Collider, generate massive files daily. As files increase in size, it is more likely errors are introduced during their transfer, such as the ones not caught by the checksum process. Reliability is especially important when the files store data that capture a rare event, such as rare gravitational waves whose loss could significantly impact scientific conclusions if they go undetected. This collaborative project brings together investigators from Arizona State University and Boston University to develop a multi-layer error detection (MLED) architecture. MLED will not only eliminate detectable errors, it also provably reduces the probability of undetected errors.Each layer in the MLED architecture is parameterized by a policy that includes the scope of the error detection method among other requirements. When aggregated, the layers improve error detection capabilities and significantly reduce the probability of undetected errors. This project has three research directions: (1) to develop the theory of MLED to study trade-offs among policy parameters to optimize the probability of undetected errors and file transfer delay; (2) to design a software-defined implementation of MLED exploiting features in FABRIC, a new testbed of networked computing systems; (3) to evaluate MLED to investigate and understand the design trade-offs that must be navigated to harness FABRIC, to validate the theory, and to compare to other tools across the spectrum of file transfer solutions. This project impacts the science of understanding networked computing systems through the design and implementation of the MLED architecture using features of FABRIC that are unavailable in other network testbeds. Reliable file transfer is especially important when the conclusions drawn from the data critically depend on their correctness, with potential consequences to scientific knowledge. In addition, MLED advances data science to better understand undetected errors, their propagation through file transfer, and management of massive data sets. FABRIC Across Borders offers the opportunity to use MLED for petabyte-scale file transfer across the oceans. Outreach activities will share experience using FABRIC with the testbed community and promote the adoption of MLED for file transfer. By the end of this project, it will have contributed towards NSF and national research priorities on networked computing systems, and helped educate graduate and undergraduate students, while broadening participation of underrepresented minorities.Project website: https://www.public.asu.edu/~syrotiuk/mled.html. Jupyter Notebooks will be used throughout the project to prototype, explore, and document the experimental process. To enable reproducibility the notebooks and code repositories will be shared on the project website, in addition to resulting publications. This website will be maintained for at least three years following the project end date.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

文件传输是互联网的基本操作。重要的科学仪器，例如詹姆斯·韦伯太空望远镜和大型强子对撞机，每天都会生成大量文件。随着文件大小的增加，在传输过程中更有可能引入错误，例如校验和过程未捕获的错误。当文件存储捕获罕见事件的数据时，可靠性尤其重要，例如罕见的引力波，如果未被发现，其丢失可能会严重影响科学结论。该合作项目汇集了亚利桑那州立大学和波士顿大学的研究人员，开发多层错误检测 (MLED) 架构。 MLED 不仅可以消除可检测到的错误，还可以降低未检测到错误的概率。MLED 架构中的每一层都通过一项策略进行参数化，其中包括错误检测方法的范围以及其他要求。 聚合后，各层可提高错误检测能力并显着降低未检测到错误的概率。该项目有三个研究方向：（1）发展MLED理论来研究策略参数之间的权衡，以优化未检测到的错误和文件传输延迟的概率； (2) 利用 FABRIC（网络计算系统的新测试平台）中的功能，设计 MLED 的软件定义实现； (3) 评估 MLED，以调查和理解利用 FABRIC 必须进行的设计权衡，验证理论，并与文件传输解决方案范围内的其他工具进行比较。该项目通过使用其他网络测试台中不可用的 FABRIC 功能来设计和实现 MLED 架构，从而影响理解网络计算系统的科学。当从数据中得出的结论严重依赖于其正确性并对科学知识产生潜在影响时，可靠的文件传输尤其重要。此外，MLED 推进了数据科学，以更好地理解未检测到的错误、它们通过文件传输的传播以及海量数据集的管理。 FABRIC Across Borders 提供了使用 MLED 进行 PB 级文件跨洋传输的机会。外展活动将与测试平台社区分享使用 FABRIC 的经验，并促进采用 MLED 进行文件传输。到该项目结束时，它将为 NSF 和国家网络计算系统的研究重点做出贡献，并帮助教育研究生和本科生，同时扩大代表性不足的少数群体的参与。项目网站：https://www.public.asu。 edu/~syrotiuk/mled.html。 Jupyter Notebooks 将在整个项目中用于原型设计、探索和记录实验过程。为了实现可重复性，除了最终的出版物之外，笔记本和代码存储库还将在项目网站上共享。该网站将在项目结束日期后维护至少三年。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。