CSR: Medium: Pythia: An Application Analysis and Online Modeling Based Prediction Framework for Scalable Resource Management

CSR：中：Pythia：基于应用分析和在线建模的可扩展资源管理预测框架

• 批准号: 1405697
• 负责人: Ali Butt
• 金额: $ 75万
• 依托单位: Virginia Polytechnic Institute and State University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2014
• 资助国家: 美国
• 起止时间: 2014-10-01 至 2020-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1405697&HistoricalAwards=false
• 关键词: CSR; Medium; Pythia; Application; Analysis

Computer applications that process large amounts of information are becoming common in a variety of science domains, such as High-Speed Physics, Economics, Genomics, Astronomy, and Meteorology. The overall goal of this project is to design software tools and technologies to support such applications efficiently on advanced computing systems. Moreover, the hardware that is used to implement such advanced systems often boasts of different types of resources, e.g., a conventional computer processor running alongside specialized graphic processing units, and this heterogeneity presents a major challenge when running the applications at the needed large scale. Having a better understanding of the applications behavior on the emerging hardware is key to sustaining these systems. To this end, the project designs and develops Pythia, software that models and predicts how applications would behave on given hardware. This information is then used to better utilize the resources, and achieve scalable and high performance computing systems.The intellectual value of this research involves three intermediate research goals. 1) Design an accurate application classifier using compile-time program analysis that captures workflow behavior and application characteristics, and provides detailed insights into expected runtime application interactions. 2) Design and develop an accurate simulation model that incorporates workflow and application characteristics into a heuristics engine to predict how the application will perform under given conditions and resources. 3) Design a distributed, flexible, efficient, and easy-to-use online oracle framework that captures the infrastructure heterogeneity and integrates with live systems to predict application behavior, which in turn can help guide application-attuned resource scheduling and management. Completion of the project will create tools and technologies for realization of more efficient and scalable computing systems. This work impacts a broad range of disciplines that regularly employ high-performance large-scale computing systems, especially for data-driven discovery. Consequently, use of Pythia will reduce the time-to-solution for modern and emerging applications, and therefore directly affect our way of life. The educational activities, which include recruiting and mentoring women and minority students, will help produce graduates with highly marketable skill sets. The integration of the research discoveries and software tools, which will be open source and made public, into the educational curriculum will help capture the interest of the next generation of computer scientists.

处理大量信息的计算机应用程序在高速物理、经济学、基因组学、天文学和气象学等各种科学领域中变得越来越普遍。该项目的总体目标是设计软件工具和技术，以在先进计算系统上有效地支持此类应用程序。此外，用于实现此类先进系统的硬件通常拥有不同类型的资源，例如与专用图形处理单元一起运行的传统计算机处理器，而这种异构性在以所需的大规模运行应用程序时提出了重大挑战。更好地了解新兴硬件上的应用程序行为是维持这些系统的关键。为此，该项目设计并开发了 Pythia，该软件可以建模并预测应用程序在给定硬件上的行为方式。然后利用这些信息更好地利用资源，实现可扩展的高性能计算系统。这项研究的智力价值涉及三个中间研究目标。 1) 使用编译时程序分析设计准确的应用程序分类器，捕获工作流行为和应用程序特征，并提供对预期运行时应用程序交互的详细见解。 2) 设计和开发精确的模拟模型，将工作流程和应用程序特征合并到启发式引擎中，以预测应用程序在给定条件和资源下的执行情况。 3）设计一个分布式、灵活、高效、易于使用的在线预言机框架，捕获基础设施异构性并与实时系统集成以预测应用程序行为，从而有助于指导应用程序调整的资源调度和管理。该项目的完成将创建用于实现更高效和可扩展的计算系统的工具和技术。这项工作影响了经常使用高性能大规模计算系统的广泛学科，特别是数据驱动的发现。因此，使用 Pythia 将缩短现代和新兴应用程序的解决时间，从而直接影响我们的生活方式。这些教育活动包括招募和指导女性和少数民族学生，将有助于培养具有高度市场化技能的毕业生。将开源和公开的研究发现和软件工具整合到教育课程中将有助于吸引下一代计算机科学家的兴趣。

项目成果

Slimmer: Weight Loss Secrets for Docker Registries

Simmer：Docker 注册表的减肥秘密

• 发表时间: 2019-07
• 期刊: Work-in-Progress report in the IEEE International Conference on Cloud Computing (CLOUD
• 作者: Zhao, Nannan;Anwar, Ali;Tarasov, Vasily;Rupprecht, Lukas;Skourtis, Dimitrios;Warke, Amit S;Mohamed, Mohamed;Butt, Ali R.
• 通讯作者: Butt, Ali R.

BESPOKV: Application Tailored Scale-Out Key-Value Stores.

BESPOKV：应用程序定制的横向扩展键值存储。

• 发表时间: 2018-11
• 期刊: and Analysis (SC
• 作者: Anwar, Ali;Cheng, Yue;Huang, Hai;Lee, Dongyoon;Han, Jingoo;Sim, Hyogi;Douglis, Fred;Butt, Ali R.
• 通讯作者: Butt, Ali R.

Finding and counting tree-like subgraphs using MapReduce

使用 MapReduce 查找和计算树状子图

• DOI: 10.1109/tmscs.2017.2768426
• 发表时间: 2017-10
• 期刊: IEEE Transactions on Multi-Scale Computing Systems
• 作者: Zhao, Zhao;Chen, Langshi;Avram, Mihai;Li, Meng;Wang, Guanying;Butt, Ali;Khan, Maleq;Marathe, Madhav;Qiu, Judy;Vullikanti, Anil
• 通讯作者: Vullikanti, Anil

Large-Scale Analysis of the Docker Hub Dataset

Docker Hub 数据集的大规模分析

• DOI: 10.1109/cluster.2019.8891000
• 发表时间: 2019-09-01
• 期刊: 2019 IEEE International Conference on Cluster Computing (CLUSTER)
• 作者: Nannan Zhao;Vasily Tarasov;Hadeel Albahar;Ali Anwar;Lukas Rupprecht;Dimitrios Skourtis;Amit Warke;Mohamed Mohamed;A. Butt
• 通讯作者: A. Butt

Toward Scalable Monitoring on Large-Scale Storage for Software Defined Cyberinfrastructure

面向软件定义网络基础设施的大规模存储的可扩展监控

• 发表时间: 2017-11
• 期刊: Proceedings of the ... Petascale Data Storage Workshop
• 作者: Paul, Arnab K.;Chard, Ryan;Chard, Kyle;Tuecke, Steven;Butt, Ali R.;Foster, Ian
• 通讯作者: Foster, Ian