SI2-SSE: Collaborative Research: ADAPT: Next Generation Message Passing Interface (MPI) Library - Open MPI

SI2-SSE：协作研究：ADAPT：下一代消息传递接口 (MPI) 库 - 开放 MPI

• 批准号: 1339763
• 负责人: Edgar Gabriel
• 金额: $ 14.72万
• 依托单位: University of Houston
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2013
• 资助国家: 美国
• 起止时间: 2013-09-01 至 2016-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1339763&HistoricalAwards=false
• 关键词: SI2; SSE; Collaborative; Research; ADAPT

High-performance computing has reshaped science and industry in many areas. However, the rapid evolution at the hardware level over the last few years have been unmatched by corresponding changes at the programming paradigm level. According to the consensus of several major studies, the degree of parallelism on large systems is expected to increase by several orders of magnitude. As a result, the Message Passing Interface (MPI), which has been the de-facto standard message passing paradigm, lacks an efficient and portable way of handling today's architectures. To efficiently handle such systems, MPI implementations must adopt more asynchronous and thread-friendly behaviors to perform better than they do on today?s systems. Maintaining and further enhancing MPI, one of the most widely-used communication libraries in high-performance computing, will have a far-reaching impact beyond the scientific community, and represents a critical building block for continued advances in all areas of science and engineering.The ADAPT project enhances, hardens and modernizes the Open MPI library in the context of this ongoing revolution in processor architecture and system design. It creates a viable foundation for a new generation of Open MPI components, enabling the rapid exploration of new physical capabilities, providing greatly improved performance portability, and working toward full interoperability between classes of components. More specifically, ADAPT implements fundamental software techniques that can be used in many-core systems to efficiently execute MPI-based applications and to tolerate fail-stop process failures, at scales ranging from current large systems to the extreme scale systems that are coming soon. To improve the efficiency of Open MPI, ADAPT integrates, as a core component, knowledge about the hardware architecture, and allows all layers of the software stack full access to this information. Process placement, distributed topologies, file accesses, point-to-point and collective communications can then adapt to such topological information, providing more portability. The ADAPT team is also updating the current collective communication layer to allow for a task-based collective description contained at a group-level, which in turn adjusts to the intra and inter-node topology. Planned expansion of the current code with resilient capabilities allows Open MPI to efficiently survive hard and soft error types of failures. These capabilities can be used as building blocks for all currently active fault tolerance proposals in the MPI standard body.MPI is already one of the most relevant parallel programming models, the most important brick of most parallel applications, and one of the most critical communication pieces of most other programing models. Thus, the experience of the research team and emerging capabilities can benefit all future users of these programming standards, tools, and libraries--regardless of discipline. Any improvement in the performance and capabilities of a major MPI library such as Open MPI, has tremendous potential for an immediate and dramatic impact on the application communities. In addition to improving the time to solution for their applications, it has the potential to decrease the energy usage and maximize the performance delivered by the existing execution platforms. The scale at which the Open MPI library is used in government research institutions (including universities and national laboratories), as well as in the private sector, is a major vector for a quick impact on all scientific and engineering communities.

高性能计算在许多领域都重塑了科学和工业。但是，在过去几年中，硬件级别的快速发展是通过编程范式级别的相应变化而无法比拟的。根据几项主要研究的共识，大型系统上的并行性程度预计将增加几个数量级。结果，传递接口（MPI）的消息一直是通过范式传递范式的事实标准消息，它缺乏一种处理当今体系结构的高效且便携式的方式。为了有效地处理此类系统，MPI实现必须采用更多的异步和线程友好的行为，以比当今的系统更好地执行。维持和进一步增强MPI是高性能计算中最广泛使用的沟通库之一，将在科学界产生深远的影响，这是科学和工程各个领域持续发展的关键基础。在这场正在进行的处理器体系结构和系统设计的革命背景下，Adapt项目可以增强，硬化和现代化开放的MPI库。它为新一代开放的MPI组件创造了可行的基础，从而可以快速探索新的物理能力，从而极大地提高了性能可移植性，并致力于在组件类别之间的完全互操作性。更具体地说，Adapt实现了可以在多核系统中使用的基本软件技术，以有效地执行基于MPI的应用程序并忍受故障停止过程故障，从当前的大型系统到即将推出的极端尺度系统的规模。为了提高开放式MPI的效率，将其作为核心组件的集成在一起，了解硬件体系结构的知识，并允许软件的所有层次堆栈完全访问此信息。过程放置，分布式拓扑，文件访问，点对点和集体通信可以适应此类拓扑信息，从而提供更多的可移植性。 Adapt团队还正在更新当前的集体通信层，以允许在组级别上包含的基于任务的集体描述，该描述又可以调整为内部和节点拓扑。具有弹性功能的当前代码的计划扩展允许开放的MPI有效地生存于故障和软误差类型的失败类型。这些功能可以用作MPI标准机构中所有当前有效的容错耐受性建议的构建块。MPI已经是最相关的平行编程模型之一，是大多数并行应用中最重要的砖块之一，也是最关键的通信作品之一在大多数其他编程模型中。因此，研究团队和新兴功能的经验可以使这些编程标准，工具和库的所有未来用户受益，而没有纪律。开放MPI等主要MPI库的性能和能力的任何改进都具有对应用程序社区产生巨大影响的巨大潜力。除了提高解决方案的时间的时间外，它还有可能减少能源使用情况并最大程度地提高现有执行平台的性能。政府研究机构（包括大学和国家实验室）以及私营部门使用开放MPI图书馆的规模是对所有科学和工程社区的快速影响的主要矢量。