Collaborative Research: SHF: Small: Model-driven Design and Optimization of Dataflows for Scientific Applications

协作研究：SHF：小型：科学应用数据流的模型驱动设计和优化

• 批准号: 2331152
• 负责人: Michela Taufer
• 金额: $ 42.4万
• 依托单位: University of Tennessee Knoxville
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-10-01 至 2025-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2331152&HistoricalAwards=false
• 关键词: Collaborative; Research; SHF; Small; Model

The increasing capability of high-performance computing (HPC), cloud computing, and edge computing systems directly translates into the ability to generate more data and execute more extended analyses, thus expanding the range of natural phenomena that scientists can study using dataflows in scientific domains such as chemistry, materials sciences, molecular biology, and drug design. At the same time, the steady growth in the complexity of these dataflows also results in new challenges in the effective composition of single data tasks into scalable dataflow pipelines. This project addresses these critical challenges by developing solutions to optimize dataflow pipelines across heterogeneous resources. This project builds a broader community of HPC experts, who will have a far-reaching impact on the efficient development of dataflow pipelines supporting scientific applications. The team of researchers promotes increased participation of underrepresented students, particularly women, through mentoring students in Systers (the organization for women in Electrical Engineering and Computer Science at the University of Tennessee Knoxville). Furthermore, the researchers develop data analytics training tailored for early career professionals and share the material with the Midwest Research Computing and Data Consortium and the attendees at the bi-annual NSF/TCPP (Technical Community on Parallel Processing) workshops on parallel and distributed computing education (EduPar). This project has four main research components. First, the project defines a taxonomy of common dataflow motifs used in scientific domains, ranging from simple producer-consumer pairs to complex pipelines with multiple producers and consumers, by mapping these motifs to real scientific applications. Second, the project designs a middleware layer to handle dataflow pipelines executing on HPC, cloud, and edge resources. Third, the project develops a 2-step model for mitigating pipelines that result in data loss and inefficiencies associated with the slowdown in data production or consumption in dataflow pipelines. Finally, the project trains a broader community to utilize the taxonomy, middleware, and model to optimize real scientific applications by identifying potential bottlenecks and making necessary adjustments to maximize pipeline efficiency and accuracy, continuously monitoring and optimizing pipelines to ensure the highest quality scientific output possible.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

高性能计算（HPC）、云计算和边缘计算系统的能力不断增强，直接转化为生成更多数据和执行更多扩展分析的能力，从而扩大了科学家在科学领域使用数据流可以研究的自然现象的范围例如化学、材料科学、分子生物学和药物设计。与此同时，这些数据流复杂性的稳定增长也给将单个数据任务有效组合成可扩展的数据流管道带来了新的挑战。该项目通过开发解决方案来优化跨异构资源的数据流管道来解决这些关键挑战。该项目建立了更广泛的 HPC 专家社区，他们将对支持科学应用的数据流管道的高效开发产生深远的影响。 研究人员团队通过指导 Systers（田纳西大学诺克斯维尔大学电气工程和计算机科学领域的女性组织）的学生，促进代表性不足的学生（尤其是女性学生）的参与。此外，研究人员还为早期职业专业人员开发了数据分析培训，并与中西部研究计算和数据联盟以及两年一度的 NSF/TCPP（并行处理技术社区）并行和分布式计算教育研讨会的与会者分享了这些材料（教育帕尔）。该项目有四个主要研究部分。首先，该项目通过将这些主题映射到实际的科学应用，定义了科学领域中使用的常见数据流主题的分类，从简单的生产者-消费者对到具有多个生产者和消费者的复杂管道。其次，该项目设计了一个中间件层来处理在 HPC、云和边缘资源上执行的数据流管道。第三，该项目开发了一个两步模型，用于缓解管道导致的数据丢失和与数据流管道中数据生产或消耗减慢相关的低效率。最后，该项目培训更广泛的社区利用分类法、中间件和模型来优化真正的科学应用，方法是识别潜在的瓶颈并进行必要的调整，以最大限度地提高管道效率和准确性，持续监控和优化管道，以确保尽可能高质量的科学输出该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。