Collaborative Research: Building the Community for the Open Storage Network

协作研究：构建开放存储网络社区

• 批准号: 1747552
• 负责人: James Glasgow
• 金额: $ 43.46万
• 依托单位: University of Illinois at Urbana-Champaign
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-06-15 至 2021-05-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1747552&HistoricalAwards=false
• 关键词: Collaborative; Research; Building; Community; Open

The scientific community is facing a major challenge dealing with the increasing amount of open scientific data emerging from research projects on all scales-- from large facilities to small research labs. Over the last five years the NSF has funded more than 200 high-speed connections to the Internet-2 backbone operating at 10-100Gbps speeds. The goal of this project is to develop a prototype module for a high performance distributed storage system that extends the usability of the existing high-speed interconnects. This project is a pilot for a potential national-scale storage infrastructure for open scientific data, which at full scale could serve hundred sites and many hundreds of Petabytes. Many of the technologies associated with such a distributed system already exist; the key challenge in this project is social engineering: how can one design a simple enough yet robust storage node that can be easily replicated, is attractive for universities and research projects to adopt, is easy to manage and can support the various patterns for large scale scientific analyses?Many universities have several of the necessary pieces for Data Intensive Science in place-- reasonably sized computing clusters, a few PB of storage and even a high-speed connection-- yet performing the analyses of data intensive science is very painful and slow. Data is never there when needed, large storage systems often fail despite having massive RAID configurations, and moving data from disk-to-disk at the full network speed still requires complex skills. The project offers a broad community buy-in through the Big Data Hubs, a unique combination of skills, facilities and science challenges to test, evaluate and deploy different hardware and software combinations that can be used in the design of a much larger, national-scale system. The goal is to design and run detailed benchmarks for various test science projects requiring different combinations of data transfer, data processing and massive compute, and use the results to design and build a low-cost, scalable petascale appliance including inexpensive hardware nodes and a simple software stack that can be replicated across many universities, supercomputer centers and large NSF facilities. The proposed system could become an enormous multiplier on the existing NSF investments in high end computing and fast networks. It could also accelerate the pace of standardization of data storage across the nation. The public, open data products, often discussed in the Data Management Plans at the end of NSF proposals could find an easy-to-use home. Various educational projects could simply rely upon a robust storage infrastructure with a simple API, and build a variety of delivery services for the educational community.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

科学界面临着一个重大挑战，即从各种规模的研究项目中出现的越来越多的开放科学数据 - 从大型设施到小型研究实验室。在过去的五年中，NSF资助了以10-100Gbps速度运行的Internet-2骨干线高度连接。该项目的目的是为高性能分布式存储系统开发一个原型模块，以扩展现有的高速互连的可用性。该项目是用于开放科学数据的潜在国家规模存储基础设施的试点，该基础设施在全面可以为数百个地点和数百个之前。 与这种分布式系统相关的许多技术已经存在；该项目中的关键挑战是社会工程：如何设计一个可以轻松复制，对大学和研究项目有吸引力，易于管理的较简单但稳健的存储节点，易于管理，并且可以为大规模科学分析的各种模式提供支持？许多大学都有几个必要的大学来进行数据密集的科学，以合理地进行计算的科学，并具有一定的数据，甚至是一定的数据，甚至是一个较高的数据，甚至是较高的数据，甚至是较高的数据。痛苦和缓慢。数据在需要时永远不会存在，尽管具有大量的RARED配置，但大型存储系统通常会失败，并且以全网络速度从磁盘到磁盘移动数据仍然需要复杂的技能。该项目通过大数据中心提供了广泛的社区买入，这是测试，评估和部署不同硬件和软件组合的技能，设施和科学挑战的独特组合，这些组合可用于设计更大的国家规模系统。目的是针对各种测试科学项目设计和运行详细的基准测试，需要不同的数据传输组合，数据处理和大规模计算，并利用结果来设计和构建低成本，可扩展的Petascale设备，包括廉价的硬件节点以及可以在许多大学，超级载体中心，超级载体中心和大型NSF中复制的简单软件堆栈。提出的系统可能会成为高端计算和快速网络中现有NSF投资的巨大乘数。它还可以加速全国数据存储的标准化速度。 NSF提案结束时的数据管理计划中经常讨论的公众开放数据产品可能会找到易于使用的房屋。各种教育项目可以简单地使用简单的API依靠强大的存储基础设施，并为教育社区建立各种交付服务。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子优点和更广泛影响的评估来评估值得支持的。