SHF: CSR: Small: Toward Smart HPC through Active Learning and Intelligent Scheduling

SHF：CSR：小型：通过主动学习和智能调度迈向智能 HPC

• 批准号: 1422009
• 负责人: Zhiling Lan
• 金额: $ 49.88万
• 依托单位: Illinois Institute of Technology
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2014
• 资助国家: 美国
• 起止时间: 2014-08-15 至 2019-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1422009&HistoricalAwards=false
• 关键词: SHF; CSR; Small; Toward; Smart

As high performance computing (HPC) continues to grow in scale, energy and resilience become first-class concerns, in addition to the pursuit of performance. These concerns demand significant changes in many aspects of the system stack including resource management and job scheduling. In order to harness the great potential of extreme scale systems, this project aims to incorporate intelligence into resource management and job scheduling. More specifically, it will develop a framework named SPEaR (Scheduling for Performance, Energy, and Resilience efficiency) for dynamically optimizing the three-dimensional performance, energy, and resilience scheduling. The research focuses on two thrusts: one is active learning to automatically extract valuable performance, energy, and resilience patterns and tradeoffs out of application and system data, and the other is intelligent scheduling to improve and control performance, resilience, and energy efficiency in resource management and scheduling. An event-driven scheduling simulator is being developed for comprehensively evaluating scheduling policies and their aggregate effects. The simulator, along with system logs, will be made available to the broad community under an open source license. This project creates critical technologies to promote system productivity and makes important advances essential toward smart HPC. Additionally, the learning techniques developed in this project are useful to other big data problems of national interests. The education plan enhances the undergraduate and graduate curricula and broadens the participation from underrepresented groups.

随着高性能计算 (HPC) 规模的不断增长，除了对性能的追求之外，能源和弹性也成为首要关注的问题。这些问题需要对系统堆栈的许多方面进行重大更改，包括资源管理和作业调度。为了发挥超大规模系统的巨大潜力，该项目旨在将智能融入资源管理和作业调度中。更具体地说，它将开发一个名为 SPEaR（性能、能源和弹性效率调度）的框架，用于动态优化三维性能、能源和弹性调度。该研究重点关注两个重点：一是主动学习，从应用程序和系统数据中自动提取有价值的性能、能源和弹性模式和权衡；二是智能调度，以提高和控制资源的性能、弹性和能源效率。管理和调度。正在开发事件驱动的调度模拟器，用于综合评估调度策略及其总体效果。该模拟器以及系统日志将在开源许可下向广大社区开放。该项目创建了提高系统生产力的关键技术，并为智能 HPC 取得了至关重要的重要进步。此外，该项目中开发的学习技术对于国家利益的其他大数据问题也很有用。该教育计划加强了本科生和研究生课程，并扩大了代表性不足群体的参与。

项目成果

The Effect of System Utilization on Application Performance Variability

系统利用率对应用程序性能变化的影响

• 发表时间: 2019-06
• 期刊: ACM Digital Library
• 作者: Li, Boyang;Chunduri, Sudheer;Harms, Kevin;Fan, Yuping;Lan, Zhiling
• 通讯作者: Lan, Zhiling

Modeling and Analysis of Application Interference on Dragonfly+

Dragonfly应用干扰建模与分析

• 发表时间: 2019-06
• 期刊: ACM SIGSIM PADS
• 作者: Kang, Yao;Wang, Xin;McGlohon, Neil;Mubarak, Misbah;Chunduri, Sudheer;Lan, Zhiling
• 通讯作者: Lan, Zhiling