CSR: Medium: Pythia: An Application Analysis and Online Modeling Based Prediction Framework for Scalable Resource Management

CSR：中：Pythia：基于应用分析和在线建模的可扩展资源管理预测框架

• 批准号: 1405697
• 负责人: Ali Butt
• 金额: $ 75万
• 依托单位: Virginia Polytechnic Institute and State University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2014
• 资助国家: 美国
• 起止时间: 2014-10-01 至 2020-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1405697&HistoricalAwards=false
• 关键词: CSR; Medium; Pythia; Application; Analysis

Computer applications that process large amounts of information are becoming common in a variety of science domains, such as High-Speed Physics, Economics, Genomics, Astronomy, and Meteorology. The overall goal of this project is to design software tools and technologies to support such applications efficiently on advanced computing systems. Moreover, the hardware that is used to implement such advanced systems often boasts of different types of resources, e.g., a conventional computer processor running alongside specialized graphic processing units, and this heterogeneity presents a major challenge when running the applications at the needed large scale. Having a better understanding of the applications behavior on the emerging hardware is key to sustaining these systems. To this end, the project designs and develops Pythia, software that models and predicts how applications would behave on given hardware. This information is then used to better utilize the resources, and achieve scalable and high performance computing systems.The intellectual value of this research involves three intermediate research goals. 1) Design an accurate application classifier using compile-time program analysis that captures workflow behavior and application characteristics, and provides detailed insights into expected runtime application interactions. 2) Design and develop an accurate simulation model that incorporates workflow and application characteristics into a heuristics engine to predict how the application will perform under given conditions and resources. 3) Design a distributed, flexible, efficient, and easy-to-use online oracle framework that captures the infrastructure heterogeneity and integrates with live systems to predict application behavior, which in turn can help guide application-attuned resource scheduling and management. Completion of the project will create tools and technologies for realization of more efficient and scalable computing systems. This work impacts a broad range of disciplines that regularly employ high-performance large-scale computing systems, especially for data-driven discovery. Consequently, use of Pythia will reduce the time-to-solution for modern and emerging applications, and therefore directly affect our way of life. The educational activities, which include recruiting and mentoring women and minority students, will help produce graduates with highly marketable skill sets. The integration of the research discoveries and software tools, which will be open source and made public, into the educational curriculum will help capture the interest of the next generation of computer scientists.

处理大量信息的计算机应用在各种科学领域中变得普遍，例如高速物理，经济学，基因组学，天文学和气象。该项目的总体目标是设计软件工具和技术，以在高级计算系统上有效地支持此类应用程序。此外，用于实施此类高级系统的硬件通常具有不同类型的资源，例如，传统的计算机处理器与专业的图形处理单元一起运行，并且这种异质性在需要大规模的大规模运行应用程序时会带来重大挑战。更好地了解新兴硬件上的应用程序行为是维持这些系统的关键。为此，该项目设计和开发Pythia，该软件是建模并预测应用程序在给定硬件上的行为。然后，该信息用于更好地利用资源，并实现可扩展和高性能计算系统。该研究的智力价值涉及三个中间研究目标。 1）使用编译时计划分析设计精确的应用程序分类器，该分析符捕获工作流程行为和应用程序特征，并为预期的运行时应用程序交互提供详细的见解。 2）设计和开发一个准确的仿真模型，该模型将工作流程和应用程序特征纳入启发式引擎中，以预测应用程序在给定条件和资源下的执行方式。 3）设计一个分布式，灵活，高效且易于使用的在线Oracle框架，该框架捕获基础架构异质性并与实时系统集成以预测应用程序行为，从而可以帮助指导应用程序攻击的资源调度和管理。该项目的完成将创建工具和技术，以实现更高效，更可扩展的计算系统。这项工作影响了广泛的学科，这些学科经常采用高性能的大规模计算系统，尤其是对于数据驱动的发现。因此，使用腓热将减少现代和新兴应用的解决时间，因此直接影响我们的生活方式。教育活动包括招聘和指导妇女和少数族裔学生，将有助于培养具有高度销售技能的毕业生。研究发现和软件工具的整合将是开源的，并将其公开化为教育课程，这将有助于吸引下一代计算机科学家的兴趣。

项目成果

Toward Transparent Data Management in Multi-Layer Storage Hierarchy of HPC Systems

• DOI: 10.1109/ic2e.2018.00046
• 发表时间: 2018-05
• 期刊: 2018 IEEE International Conference on Cloud Engineering (IC2E)
• 作者: Bharti Wadhwa;S. Byna;A. Butt

iez: Resource Contention Aware Load Balancing for Large-Scale Parallel File Systems

• DOI: 10.1109/ipdps.2019.00070
• 发表时间: 2019-05
• 期刊: 2019 IEEE International Parallel and Distributed Processing Symposium (IPDPS)
• 作者: Bharti Wadhwa;A. Paul;Sarah Neuwirth;Feiyi Wang;S. Oral;A. Butt;Jon Bernard;K. Cameron

Improving I/O Performance of HPC Application Using Intra-Job Scheduling.

使用作业内调度提高 HPC 应用程序的 I/O 性能。

• 发表时间: 2019
• 期刊: Proceedings of the 4th Joint International Workshop on Parallel Data Storage & Data Intensive Scalable Computing Systems (PDSW-DISC'19
• 作者: Paul, Arnab K.';Moody, Adam;Gonsiorowski, Elsa;Mohror, Kathryn;Butt, Ali R.
• 通讯作者: Butt, Ali R.

MOANA: Modeling and Analyzing I/O Variability in Parallel System Experimental Design

MOANA：并行系统实验设计中 I/O 可变性的建模和分析

• DOI: 10.1109/tpds.2019.2892129
• 发表时间: 2019
• 期刊: IEEE Transactions on Parallel and Distributed Systems
• 影响因子: 5.3
• 作者: Cameron, Kirk W.;Anwar, Ali;Cheng, Yue;Xu, Li;Li, Bo;Ananth, Uday;Bernard, Jon;Jearls, Chandler;Lux, Thomas;Hong, Yili
• 通讯作者: Hong, Yili

Toward scalable monitoring on large-scale storage for software defined cyberinfrastructure

• DOI: 10.1145/3149393.3149402
• 发表时间: 2017-11
• 期刊: Proceedings of the 2nd Joint International Workshop on Parallel Data Storage & Data Intensive Scalable Computing Systems
• 作者: A. Paul;S. Tuecke;Ryan Chard;A. Butt;K. Chard;Ian T Foster