Collaborative Research: Elements: Simulation-driven Evaluation of Cyberinfrastructure Systems

协作研究：要素：网络基础设施系统的仿真驱动评估

• 批准号: 2103489
• 负责人: Henri Casanova
• 金额: $ 28.5万
• 依托单位: University of Hawaii
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-08-01 至 2024-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2103489&HistoricalAwards=false
• 关键词: Collaborative; Research; Elements; Simulation; driven

Most scientific breakthroughs and discoveries are now preconditioned on performing complex processing of vast amounts of data as conveniently, reliably, and efficiently as possible. This requires high-end interconnected compute and storage resources, as well as software systems to automate the processing on these resources. An enormous amount of effort has been invested in producing such "cyberinfrastructure" software systems. And yet, developing and evolving these systems so that they are as efficient as possible, while anticipating future cyberinfrastructure opportunities and needs, is an open challenge. This project transforms the way in which these systems are evaluated, so that their capabilities can be developed and evolved judiciously. The traditional evaluation approach is to observe executions of these systems on real-world hardware resources. Although seemingly natural, this approach suffers from many shortcomings. Instead, this project focuses on simulating these executions. Simulation has tremendous, and untapped, potential for transforming the development cycle of cyberinfrastructure systems. Specifically, this project produces software elements that can be easily integrated into existing and future systems to afford them with simulation capabilities. These capabilities make it possible for developers to put their systems through the wringer and observe their behaviors for arbitrary operating conditions, including ones that go beyond current hardware platforms and scientific applications. Simply put, these capabilities will make it possible to establish a solid experimental science approach for the development of cyberinfrastructure systems that support current and future scientific endeavors that are critical to the development of our society.The cyberinfrastructure has been the object of intensive research and development, resulting in a rich set of interoperable software systems that are used to support science. A key challenge is the development of systems that can execute application workloads efficiently, while anticipating future cyberinfrastructure opportunities and needs. This project aims to transform the way in which these systems are evaluated, so that their capabilities can be evolved based on a sound, quantitative experimental science approach. The traditional evaluation approach is to use full-fledged software stacks to execute application workloads on actual cyberinfrastructure deployments. Unfortunately, this approach suffers from several shortcomings: real-world experiments are time- and labor-intensive, and they are limited to currently available hardware and software configurations. An alternative to real-world experiments that does not suffer from these shortcomings is simulation, i.e., the implementation and use of a software artifact that models the functional and performance behaviors of software and hardware stacks of interest. This project uses simulation to transform the way in which cyberinfrastructure systems are evaluated as part of their long-term development cycles. This is achieved via software elements for enhancing production cyberinfrastructure systems with simulation capabilities so as to enable quantitative evaluation of these systems for arbitrary execution scenarios. Creating these scenarios requires little labor, and executions can be simulated accurately and orders of magnitude faster than their real-world counterparts. Furthermore, simulations are perfectly reproducible and observable. While this approach is general, its effectiveness will be demonstrated by applying it to a number of production systems, namely, workflow management systems. This project capitalizes on the years of development invested in the SimGrid and WRENCH simulation frameworks.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

现在，大多数科学的突破和发现都针对尽可能方便，可靠和有效地进行大量数据进行复杂处理。这需要高端互连的计算和存储资源以及软件系统来自动化这些资源的处理。已经投入了大量精力来生产此类“网络基础设施”软件系统。然而，开发和不断发展这些系统，以便它们尽可能高效，同时预期未来的网络基础设施机会和需求，这是一个公开的挑战。 该项目改变了对这些系统的评估方式，以便可以明智地发展和发展它们的能力。传统的评估方法是观察这些系统在现实世界硬件资源上的执行。尽管看似很自然，但这种方法遭受了许多缺点。相反，该项目着重于模拟这些执行。模拟具有巨大且未开发的潜力，可以改变网络基础结构系统的发展周期。具体而言，该项目生产的软件元素可以轻松地集成到现有和将来的系统中，以提供模拟功能。 这些功能使开发人员有可能将其系统置于扭曲器中，并观察其对任意操作条件的行为，包括超越当前硬件平台和科学应用程序的行为。简而言之，这些能力将使建立一种可靠的实验科学方法来开发网络基础设施系统，以支持当前和未来的科学努力，这对于我们社会的发展至关重要。Cyber​​infrstructure一直是强化研究和开发的对象，导致了一系列可用的软件系统来支持科学。一个关键的挑战是可以开发可以有效执行应用程序工作负载的系统，同时预测未来的网络基础设施机会和需求。该项目旨在改变评估这些系统的方式，以便可以根据声音，定量的实验科学方法来发展其功能。传统的评估方法是使用成熟的软件堆栈在实际的网络基础结构部署上执行应用程序工作负载。不幸的是，这种方法遇到了几个缺点：现实世界实验是时间和劳动力密集的，并且仅限于当前可用的硬件和软件配置。不受这些缺点的现实世界实验的替代方法是模拟，即，对软件和性能行为的实现和使用，对软件的功能和性能行为进行了建模。该项目使用仿真来改变网络基础结构系统作为其长期发展周期的一部分的方式。 这是通过软件元素实现的，用于增强具有模拟功能的生产网络基础结构系统，以便对这些系统进行定量评估以进行任意执行方案。创建这些方案几乎不需要劳动，并且可以准确地模拟执行，并且比实际世界对应的速度更快。此外，模拟是完全可复制和可观察的。尽管这种方法是一般性的，但它的有效性将通过将其应用于许多生产系统（即工作流管理系统）来证明。 该项目利用了投资于Simgrid和Wrench模拟框架的多年发展。该奖项反映了NSF的法定任务，并使用基金会的知识分子优点和更广泛的影响审查标准，被认为值得通过评估来获得支持。

项目成果

Emerging Frameworks for Advancing Scientific Workflows Research, Development, and Education

推进科学工作流程研究、开发和教育的新兴框架

• DOI: 10.1109/works54523.2021.00015
• 发表时间: 2021
• 期刊: 2021 IEEE Workshop on Workflows in Support of Large-Scale Science (WORKS
• 作者: Casanova, Henri;Deelman, Ewa;Gesing, Sandra;Hildreth, Michael;Hudson, Stephen;Koch, William;Larson, Jeffrey;McDowell, Mary Ann;Meyers, Natalie;Navarro, John-Luke
• 通讯作者: Navarro, John-Luke

WfCommons: Data Collection and Runtime Experiments using Multiple Workflow Systems

WfCommons：使用多个工作流系统的数据收集和运行时实验

• 发表时间: 2023
• 期刊: 1st IEEE International Workshop on Workflows in Distributed Environments (WiDE
• 作者: Casanova, H.;K. Berney, K.;Chastel, S.;R. Ferreira da Silva, Rafael
• 通讯作者: R. Ferreira da Silva, Rafael

Automated generation of scientific workflow generators with WfChef

使用 WfChef 自动生成科学工作流程生成器

• DOI: 10.1016/j.future.2023.04.031
• 发表时间: 2023
• 期刊: Future Generation Computer Systems
• 作者: Coleman, Tainã;Casanova, Henri;Ferreira da Silva, Rafael
• 通讯作者: Ferreira da Silva, Rafael

WfBench: Automated Generation of Scientific Workflow Benchmarks

• DOI: 10.1109/pmbs56514.2022.00014
• 发表时间: 2022-10
• 期刊: 2022 IEEE/ACM International Workshop on Performance Modeling, Benchmarking and Simulation of High Performance Computer Systems (PMBS)
• 作者: T. Coleman;H. Casanova;K. Maheshwari;L. Pottier;Sean R. Wilkinson;J. Wozniak;F. Suter;M. Shankar;Rafael Ferreira da Silva

WfCommons: A framework for enabling scientific workflow research and development

• DOI: 10.1016/j.future.2021.09.043
• 发表时间: 2021-10-13
• 期刊: FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE
• 影响因子: 7.5
• 作者: Coleman, Taina;Casanova, Henri;da Silva, Rafael Ferreira
• 通讯作者: da Silva, Rafael Ferreira