SHF:Small:Intelligent Management of Hybrid Workloads for Extreme Scale Computing

SHF:Small：超大规模计算混合工作负载的智能管理

• 批准号: 2109316
• 负责人: Zhiling Lan
• 金额: $ 50万
• 依托单位: Illinois Institute of Technology
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-10-01 至 2024-01-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2109316&HistoricalAwards=false
• 关键词: SHF; Small; Intelligent; Management; Hybrid

The high-performance computing (HPC) community is embracing artificial intelligence (AI) techniques for countless pursuits, from driving ground-breaking scientific discoveries to protecting our national security. As newly emerging machine learning and date-centric workloads proliferate in HPC, current workload-management systems cannot keep up with the significant challenges introduced by the diverse mix of applications co-running on heterogeneous systems. This project tackles the problem by developing new workload-management methods to catalyze the convergence of HPC, AI, and data analytics. It will develop fundamental improvements in HPC workload management to promote the use of large-scale supercomputers for emerging data-centric applications (HPC4AI). Meanwhile it will exploit advanced AI technologies, especially multi-objective reinforcement learning, to empower job scheduling and resource allocation in HPC (AI4HPC). The project aims to develop an intelligent workload-management framework named MINT in which distinctive computational resource requirements of hybrid workloads will be automatically identified and fulfilled to achieve extreme resource efficiency and satisfactory user experience. Key research thrusts are: understanding performance implications of diverse workloads on supercomputers via model-driven analysis; new intelligent multi-resource scheduling methods; smart resource-allocation strategies for minimal workload interference; and extensive evaluation of the proposed framework through trace-based simulation and testing. The deliverables include a new workload-management framework and open-source software releases for intelligent management of hybrid workloads on extreme-scale systems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

高性能计算 (HPC) 社区正在拥抱人工智能 (AI) 技术来实现无数的追求，从推动突破性的科学发现到保护我们的国家安全。随着高性能计算中新兴的机器学习和以数据为中心的工作负载激增，当前的工作负载管理系统无法应对异构系统上共同运行的多种应用程序组合带来的重大挑战。该项目通过开发新的工作负载管理方法来促进 HPC、AI 和数据分析的融合来解决这个问题。它将对 HPC 工作负载管理进行根本性改进，以促进大型超级计算机在新兴的以数据为中心的应用程序 (HPC4AI) 中的使用。同时，它将利用先进的人工智能技术，特别是多目标强化学习，为HPC（AI4HPC）中的作业调度和资源分配赋能。 该项目旨在开发一个名为 MINT 的智能工作负载管理框架，其中混合工作负载的独特计算资源需求将被自动识别和满足，以实现极高的资源效率和令人满意的用户体验。 主要研究重点是：通过模型驱动分析了解超级计算机上不同工作负载的性能影响；新型智能多资源调度方法；智能资源分配策略，最大限度地减少工作负载干扰；并通过基于跟踪的模拟和测试对所提出的框架进行广泛评估。可交付成果包括新的工作负载管理框架和开源软件版本，用于对超大规模系统上的混合工作负载进行智能管理。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力优势和更广泛的影响进行评估，被认为值得支持审查标准。

项目成果

Generating Topic-Preserving Synthetic News

生成保留主题的综合新闻

• DOI: 10.1109/bigdata52589.2021.9671623
• 发表时间: 2021-12-15
• 期刊: 2021 IEEE International Conference on Big Data (Big Data)
• 作者: Ahmadreza Mosallanezhad;Kai Shu;Huan Liu
• 通讯作者: Huan Liu

Hybrid Workload Scheduling on HPC Systems

HPC 系统上的混合工作负载调度

• DOI: 10.1109/ipdps53621.2022.00052
• 发表时间: 2022-05
• 期刊: 2022 IEEE International Parallel and Distributed Processing Symposium (IPDPS
• 作者: Fan, Yuping;Lan, Zhiling;Rich, Paul;Allcock, William;Papka, Michael E.
• 通讯作者: Papka, Michael E.

Domain Adaptive Fake News Detection via Reinforcement Learning

通过强化学习进行领域自适应假新闻检测

• DOI: 10.1145/3485447.3512258
• 发表时间: 2022-02-16
• 期刊: Proceedings of the ACM Web Conference 2022
• 作者: Ahmadreza Mosallanezhad;Mansooreh Karami;Kai Shu;M. Mancenido;Huan Liu
• 通讯作者: Huan Liu