CCRI: ENS: Collaborative Research: Open Computer System Usage Repository and Analytics Engine

CCRI：ENS：协作研究：开放计算机系统使用存储库和分析引擎

• 批准号: 2016704
• 负责人: Saurabh Bagchi
• 金额: $ 118.39万
• 依托单位: Purdue University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-10-01 至 2023-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2016704&HistoricalAwards=false
• 关键词: CCRI; ENS; Collaborative; Research; Open

In science and engineering research, large-scale, centrally managed computing clusters or “supercomputers” have been instrumental in enabling the kinds of resource-intensive simulations, analyses, and visualizations that have been used in computer-aided drug discovery, high strength materials design for cars and jet engines, and disease vector analysis to name a few. Such clusters are complex systems comprised of several hundred to thousand computer servers with fast network connections between them, various data storage resources, and highly optimized scientific software being shared with several hundred other researchers from diverse domains. Consequently, the overall dependability of such systems relies on the dependability of these individual highly interconnected elements as well as the characteristics of cascading failures. While computer systems researchers and practitioners have been at the forefront of designing and deploying dependable computing cluster systems, this task has been hampered by the lack of publicly available, real-world failure data from supercomputers currently in operation. Prior practice has largely involved tedious, manual collection and curation of small sets of data for use in specific analyses. This project will establish seamless, automated pipelines for acquiring, processing, and curating continuous, detailed system usage, monitoring, and failure data from large computing clusters at two organizations, Purdue University and the University of Texas at Austin. This data will be disseminated through a publicly accessible portal and complemented by a suite of in-situ analytics capabilities that will support and spur research in dependable computing systems. The data acquisition pipeline and analytics software will be made open-source and designed for ease of federation, extension, and adoption to cluster systems operated by other organizations.Cluster computing systems are a key resource in time-sensitive, computationally intensive research such as virus structure modeling and drug discovery and have been at the forefront of efforts to tackle global pandemics. Both unanticipated system down-times and lack of actionable feedback to researchers on computational failures can have adverse effects on research timeliness and efficiency. This project will allow the practitioners and administrators of these systems to develop data-backed best practices for ensuring high availability and utilization for their clusters. The resulting large, public data repository consisting of data from clusters with diverse workloads spanning traditional high-performance computing, modern accelerator-based computing (for example on graphics processing units (GPUs)), and cloud-style applications will allow the systems research community to consider forward-looking research questions based on real system data. The project will train a cadre of students in data analysis on live production systems and this will provide them with a unique learning experience, interfacing with a variety of stakeholders.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在科学和工程研究中，大型，中央管理的计算簇或“超级计算机”有助于实现在计算机辅助药物发现中使用的各种资源密集型模拟，分析和可视化，用于汽车和喷气发动机的高强度材料设计以及疾病媒介分析的高强度材料设计，以命名一些。这样的群集是复杂的系统完成了数百万千名计算机服务器，它们之间的网络连接，各种数据存储资源以及高度优化的科学软件与来自不同领域的其他数百位研究人员共享。因此，此类系统的总体可靠性取决于这些个人高度相互联系的要素的可靠性以及级联故障的特征。尽管计算机系统的研究人员和从业人员一直处于设计和部署可靠的计算集群系统的最前沿，但由于缺乏目前正在运行的超级计算机的公开可用的现实世界故障数据，该任务受到了阻碍。先前的实践在很大程度上涉及，手动收集和策划少量数据用于特定分析。该项目将建立无缝的，自动化的管道，用于获取，处理和策划连续的，详细的系统使用，监视和失败数据，来自两个组织，普渡大学和德克萨斯大学奥斯汀分校的大型计算集群。这些数据将通过公共访问的门户传播，并由一套原位分析功能完成，这些功能将支持和刺激可靠的计算系统中的研究。数据采集​​管道和分析软件将被制造开源，并设计用于易于联邦，扩展和对其他组织运营的集群系统的采用。集群计算系统是时间敏感的，计算强度的研究中的关键资源，例如病毒结构建模和药物发现，并且一直在努力攻击全球Pandem Pandemics。意想不到的系统下降和缺乏对研究人员对计算故障的可行反馈都可能对研究及时性和效率产生不利影响。该项目将使这些系统的实践者和管理人员能够开发出数据支持的最佳实践，以确保其集群的高可用性和利用率。由此产生的大型公共数据存储库由来自潜水员工作负载的群集的数据组成，这些数据涵盖了传统的高性能计算，基于现代加速器的计算（例如，图形处理单元（GPU）（GPU）和云式应用程序以及系统研究社区将允许基于实际系统数据的前瞻性研究问题。该项目将在实时生产系统的数据分析中培训一群学生，这将为他们提供独特的学习经验，与各种利益相关者接触。该奖项反映了NSF的法定任务，并通过使用基金会的知识分子优点和更广泛的影响标准通过评估来诚实地支持支持。

项目成果

ORION and the Three Rights: Sizing, Bundling, and Prewarming for Serverless DAGs

ORION 和三项权利：无服务器 DAG 的规模调整、捆绑和预热

• 发表时间: 2022
• 期刊: 16th USENIX Symposium on Operating Systems Design and Implementation (OSDI 22
• 作者: Mahgoub, Ashraf;Yi, Edgardo Barsallo;Shankar, Karthick;Elnikety, Sameh;Chaterji, Somali;Bagchi, Saurabh
• 通讯作者: Bagchi, Saurabh

Root Cause Analysis of Failures in Microservices through Causal Discovery

• 发表时间: 2022
• 作者: Azam Ikram;Sarthak Chakraborty;Subrata Mitra;S. Saini;S. Bagchi;Murat Kocaoglu

AutoForecast: Automatic Time-Series Forecasting Model Selection

• DOI: 10.1145/3511808.3557241
• 发表时间: 2022-01-01
• 期刊: PROCEEDINGS OF THE 31ST ACM INTERNATIONAL CONFERENCE ON INFORMATION AND KNOWLEDGE MANAGEMENT, CIKM 2022
• 作者: Abdallah, Mustafa;Rossi, Ryan;Bagchi, Saurabh
• 通讯作者: Bagchi, Saurabh

Closing-the-Loop: A Data-Driven Framework for Effective Video Summarization

• DOI: 10.1109/ism.2020.00042
• 发表时间: 2020-12
• 期刊: 2020 IEEE International Symposium on Multimedia (ISM)
• 作者: Ran Xu;Haoliang Wang;Stefano Petrangeli;Viswanathan Swaminathan;S. Bagchi

SONIC: Application-aware Data Passing for Chained Serverless Applications

• 发表时间: 2021
• 作者: Ashraf Y. Mahgoub;K. Shankar;S. Mitra;Ana Klimovic;S. Chaterji;S. Bagchi