CSR: Small: IRON: Reducing Workload Interference on Massively Parallel Platforms

CSR：小：IRON：减少大规模并行平台上的工作负载干扰

• 批准号: 1717763
• 负责人: Zhiling Lan
• 金额: $ 49.72万
• 依托单位: Illinois Institute of Technology
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-09-01 至 2022-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1717763&HistoricalAwards=false
• 关键词: CSR; Small; IRON; Reducing; Workload

Interconnection networks based on dragonfly and fat tree designs are becoming increasingly dominant in high-performance computing facilities and data centers. A key challenge of managing such networks is workload interference. In a multi-user computing environment, interference among applications for shared network resources can cause a vicious cycle of events (workload interference, low productivity, selfish user behavior, and poor scheduling) aggravating each other. This project seeks to tackle the vicious circle of workload interference.This project aims to develop a software framework to systematically analyze and mitigate workload interference on dragonfly and fat tree systems. Completion of the project will create novel interference-aware scheduling policies and scalable software tools for interference analysis and reduction on dragonfly and fat tree networks. The resulting data and tools collected from simulations and experiments will be made available to the broad community under an open source license. An integrated education and outreach plan will enhance the Computer Science curriculum, broaden the participation by underrepresented groups, and outreach to the surrounding communities that are predominantly African-American and Latino.The proposed research focuses on three inter-related research thrusts: (1) high-fidelity network simulation to gain insights into network interference among applications, (2) interference aware scheduling to avoid or mitigate network contention among applications, and (3) real-world experiments to quantitatively measure workload interference of representative applications on production systems and assess interference aware scheduling. The resulting IRON will provide an integrated interference analysis and reduction framework for advancing the research of workload interference on massively parallel platforms. The project website (http://www.cs.iit.edu/~lan/iron.html) provides all the information about the project (publications, software tools, data, etc.). This project repository will be maintained for at least five years after the project completion.

基于蜻蜓和胖树设计的互连网络在高性能计算设施和数据中心中日益占据主导地位。管理此类网络的一个关键挑战是工作负载干扰。在多用户计算环境中，应用程序之间对共享网络资源的干扰可能会导致事件的恶性循环（工作负载干扰、低生产率、自私的用户行为和糟糕的调度），并相互加剧。该项目旨在解决工作负载干扰的恶性循环。该项目旨在开发一个软件框架来系统地分析和减轻蜻蜓和胖树系统上的工作负载干扰。该项目的完成将创建新颖的干扰感知调度策略和可扩展的软件工具，用于分析和减少蜻蜓和胖树网络的干扰。从模拟和实验中收集的数据和工具将在开源许可下向广大社区提供。综合教育和推广计划将加强计算机科学课程，扩大代表性不足群体的参与，并推广到以非裔美国人和拉丁裔为主的周边社区。拟议的研究重点关注三个相互关联的研究重点：(1)高保真网络模拟，以深入了解应用程序之间的网络干扰，(2) 干扰感知调度，以避免或减轻应用程序之间的网络争用，以及 (3) 真实世界实验，以定量测量生产系统上代表性应用程序的工作负载干扰并评估干扰感知调度。由此产生的 IRON 将提供一个集成的干扰分析和减少框架，以推进大规模并行平台上工作负载干扰的研究。 项目网站（http://www.cs.iit.edu/~lan/iron.html）提供了有关该项目的所有信息（出版物、软件工具、数据等）。该项目存储库将在项目完成后至少维护五年。

项目成果

Union: An Automatic Workload Manager for Accelerating Network Simulation

Union：用于加速网络仿真的自动工作负载管理器

• DOI: 10.1109/ipdps47924.2020.00089
• 发表时间: 2020-05
• 期刊: 2020 IEEE International Parallel and Distributed Processing Symposium (IPDPS
• 作者: Wang, Xin;Mubarak, Misbah;Kang, Yao;Ross, Robert B.;Lan, Zhiling
• 通讯作者: Lan, Zhiling

Trade-Off Study of Localizing Communication and Balancing Network Traffic on a Dragonfly System

Dragonfly 系统上本地化通信和平衡网络流量的权衡研究

• 发表时间: 2018-05
• 期刊: IPDPS .... [proceedings]
• 作者: Wang, Xin;Mubarak, Misbah;Yang, Xu;Ross, Rob;Lan, Zhiling
• 通讯作者: Lan, Zhiling

DRAS-CQSim: A reinforcement learning based framework for HPC cluster scheduling

DRAS-CQSim：基于强化学习的 HPC 集群调度框架

• DOI: 10.1016/j.simpa.2021.100077
• 发表时间: 2021-04
• 期刊: Software impacts
• 作者: Fan, Yuping;Lan, Zhiling
• 通讯作者: Lan, Zhiling

Modeling and Analysis of Application Interference on Dragonfly+

Dragonfly应用干扰建模与分析

• 发表时间: 2019-06
• 期刊: ACM SIGSIM PADS
• 作者: Kang, Yao;Wang, Xin;McGlohon, Neil;Mubarak, Misbah;Chunduri, Sudheer;Lan, Zhiling
• 通讯作者: Lan, Zhiling

Scheduling Beyond CPUs for HPC

超越 CPU 的 HPC 调度

• DOI: 10.1145/3307681.3325401
• 发表时间: 2019-06-17
• 期刊: Proceedings of the 28th International Symposium on High-Performance Parallel and Distributed Computing
• 作者: Yuping Fan;Z. Lan;Paul M. Rich;W. Allcock;M. Papka;Brian Austin;D. Paul
• 通讯作者: D. Paul