Collaborative Research: 100G Connectivity for Data-Intensive Computing at JHU

合作研究：JHU 数据密集型计算的 100G 连接

• 批准号: 1137045
• 负责人: Alexander Szalay
• 金额: $ 96.13万
• 依托单位: Johns Hopkins University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2012
• 资助国家: 美国
• 起止时间: 2012-01-01 至 2015-12-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1137045&HistoricalAwards=false
• 关键词: Collaborative; Research; 100G; Connectivity; Data

The project establishes 100G connectivity to the Johns Hopkins University in support of data intensive science. Recently, the NSF has funded several regional centers to bring their connection speed to Internet2 and the NLR to 100G. In Oct 2011 DOE and Internet 2 have announced the world's first transcontinental deployment of a 100G network based on coherent technology. Connections are now operational between (among others) New York, Washington DC, Chicago, and Sunnyvale. This comprehensive transformation includes the MidAtlantic Crossroads (MAX) that provides the JHU connectivity to Internet2.Johns Hopkins is also in the process of establishing its own high-speed data overlay research network across the University, connecting 6 locations across the university with multiple 10G connections. These would be aggregated into a single 100G outgoing path to MAX and beyond: to the Teragrid, Internet2, and other national resources supporting computational science.JHU has been awarded an NSF MRI grant to build the 5PB Data-Scope, a novel instrument to observe large data sets. The system will not only have large storage, but it will have extreme IO bandwidth (450GBps) and GPU based processing capability (~200TF), aimed at highly data-intensive analyses simply not possible anywhere else today. Data sets to be analyzed include very large simulations from astrophysics (1PB), ocean circulation models (600TB), computational fluid dynamics (300TB), bioinformatics and neuroscience (2-300TB each). The main difficulty in these challenges is how to bring the data sets to the Data-Scope -- most of these are generated externally, like on the Teragrid, or at the Oak Ridge Jaguar and Kraken systems.These amounts of data are pushing the limits of even a 10G connection. Demonstrating the ability to move PB of data and analyzing them in a timely fashion would encourage others to follow, and would change the way large data problems are approached in all areas of science today. It would also connect the Data-Scope to the whole US community, who can explore how to use such an instrument. It can also serve as a local hub for aggregating data sets for fast uploads into other data intensive national facilities, like the Gordon system in San Diego and the Open Science Data Grid, centered in Chicago. Without the high speed network it will be quite challenging to move petabytes: transferring a 1PB data set at an effective 5Gbps would take 18.5 days. Moving data at 100Gbps would shrink the time to 1 day, a make or break difference for a PB.In order to upgrade to 100G the award activities provide end equipment at MAX, the corresponding optics at JHU, Cisco optics, the addition of a 100G capable card to the Cisco Nexus 7K switch at the JHU side, and a 6x40G-based card for the Nexus 7K to link to the internal JHU research network hubs around the University, where data-intensive resources (clusters, instruments, storage) are located. This strategy would enable the delivery of large amounts of data to the most critical locations at JHU.The proposed high-speed connectivity would enable JHU and its partners to move petabyte-scale data sets, and enable a wide community to tackle cutting edge problems, from the traditional HPC and CFD to people who study the connectivity of the Internet. Students and researchers could do research on topics involving very large datasets, a very current and timely area, and acquire data management and analysis skills on a scale previously unheard of in a university setting.

该项目与约翰·霍普金斯大学建立 100G 连接，以支持数据密集型科学。最近，NSF 资助了多个区域中心，将其 Internet2 和 NLR 的连接速度提高到 100G。 2011 年 10 月，DOE 和 Internet 2 宣布全球首个基于相干技术的 100G 网络跨大陆部署。目前，纽约、华盛顿特区、芝加哥和桑尼维尔之间的航班已开通。这一全面转型包括为约翰霍普金斯大学提供 Internet2 连接的中大西洋十字路口 (MAX)。约翰·霍普金斯大学还在整个大学内建立自己的高速数据覆盖研究网络，通过多个 10G 连接将大学内的 6 个地点连接起来。这些将被聚合成一条单一的 100G 传出路径，到达 MAX 及其他：Teragrid、Internet2 和其他支持计算科学的国家资源。JHU 已获得 NSF MRI 拨款，用于构建 5PB Data-Scope，这是一种新颖的观测仪器大数据集。该系统不仅具有大存储空间，而且具有极高的 IO 带宽 (450GBps) 和基于 GPU 的处理能力 (~200TF)，旨在进行当今其他任何地方都无法实现的高度数据密集型分析。待分析的数据集包括来自天体物理学 (1PB)、海洋环流模型 (600TB)、计算流体动力学 (300TB)、生物信息学和神经科学（各 2-300TB）的超大型模拟。这些挑战的主要困难是如何将数据集引入数据范围——其中大部分是外部生成的，例如在 Teragrid 上，或者在 Oak Ridge Jaguar 和 Kraken 系统上。这些数据量正在突破极限。甚至 10G 连接。 展示移动 PB 数据并及时分析它们的能力将鼓励其他人效仿，并将改变当今所有科学领域处理大数据问题的方式。它还会将 Data-Scope 连接到整个美国社区，让他们可以探索如何使用这种工具。它还可以充当聚合数据集的本地中心，以便快速上传到其他数据密集型国家设施，例如圣地亚哥的戈登系统和以芝加哥为中心的开放科学数据网格。如果没有高速网络，传输 PB 级数据将非常具有挑战性：以有效 5Gbps 的速度传输 1PB 数据集将需要 18.5 天。以 100Gbps 的速度移动数据会将时间缩短至 1 天，这对于 PB 来说是成败的区别。为了升级到 100G，奖励活动提供了 MAX 的终端设备、JHU 的相应光学器件、思科光学器件、添加 100G连接到 JHU 侧 Cisco Nexus 7K 交换机的功能卡，以及用于 Nexus 7K 的基于 6x40G 的卡，以链接到周围的内部 JHU 研究网络集线器大学，数据密集型资源（集群、仪器、存储）所在的地方。该战略将使大量数据能够传输到 JHU 最关键的位置。拟议的高速连接将使 JHU 及其合作伙伴能够移动 PB 级数据集，并使广泛的社区能够解决前沿问题，从传统的 HPC 和 CFD 到研究互联网连接的人们。学生和研究人员可以对涉及非常大的数据集、一个非常当前和及时的领域的主题进行研究，并以大学环境中闻所未闻的规模获得数据管理和分析技能。