CSR: Small: Collaborative Research: Tuning Extreme-scale Storage Stack through Deep Reinforcement Learning

CSR：小型：协作研究：通过深度强化学习调整超大规模存储堆栈

• 批准号: 1817094
• 负责人: Yong Chen
• 金额: $ 24万
• 依托单位: Texas Tech University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1817094&HistoricalAwards=false
• 关键词: CSR; Small; Collaborative; Research; Tuning

Many research domains, such as high-energy physics, climate science, astrophysics, combustion science, and computational biology, need to process large amounts of data. Such domains are heavily relying on the capabilities of high performance computing (HPC) systems to manage and efficiently process massive amounts of data. Consequently, applications in the aforementioned research domains require highly optimized performance on the HPC storage systems that store, manage, and manipulate data. This project aims to utilize deep reinforcement learning methods to fine-tune the HPC storage system for optimized performance.This research explores the feasibility of leveraging deep reinforcement learning to optimize HPC storage systems by: (a) Creating a deep learning based HPC storage stack model; (b) Remodeling existing HPC storage stack to support automated configuration and tuning; (c) Collecting training datasets and training the storage stack model; and (d) utilizing the model as a responsive and playable virtual environment to learn the best policy to tune parameters. As a collaborative project, this research aims to advance the domain knowledge of both HPC storage systems and machine learning. The enhanced performance on the HPC storage stack will in turn benefit scientific discovery and thus our society. The investigators will integrate research, education, and outreach efforts during the course of this project, including recruiting and retaining of underrepresented students, mentoring graduate and undergraduate students, integrating research findings into curriculum, and publishing and disseminating results.The data collected to train the storage stack model will be shared at https://discl.cs.ttu.edu/tuningstorage while the code of machine learning at https://github.com/forrestbao/DL4SC. Results and data will be made available by the time of publication. The data will be annotated as appropriate to facilitate interpretation. The principal investigators will strive to maintain the repositories as long as possible.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

许多研究领域，例如高能物理学，气候科学，天体物理学，燃烧科学和计算生物学，都需要处理大量数据。这些领域在很大程度上依赖于高性能计算（HPC）系统的功能来管理和有效处理大量数据。因此，上述研究域中的应用需要在存储，管理和操纵数据的HPC存储系统上进行高度优化的性能。该项目旨在利用深厚的强化学习方法来微调HPC存储系统以优化性能。本研究探讨了利用深厚的增强学习以通过以下方式优化HPC存储系统的可行性： ; （b）重塑现有的HPC存储堆栈以支持自动配置和调整； （c）收集培训数据集并培训存储堆栈模型； （d）利用模型作为响应且可播放的虚拟环境来学习调整参数的最佳策略。作为一个协作项目，这项研究旨在促进HPC存储系统和机器学习的领域知识。 HPC存储堆栈的增强性能反过来将受益于科学发现，从而使我们的社会受益。调查人员将在该项目过程中整合研究，教育和外展工作，包括招募和保留代表性不足的学生，指导研究生和本科生，将研究结果集成到课程中，以及发布和传播结果。存储堆栈模型将在https://discl.cs.ttu.edu/tuningstorage上共享，而机器学习代码在https://github.com/forrestbao/dl4sc上。结果和数据将在发布时提供。数据将适当地注释以促进解释。首席调查人员将尽可能长时间地维持存储库。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子优点和更广泛影响的审查标准来评估值得支持的。

项目成果

Data Distribution for Heterogeneous Storage Systems

• DOI: 10.1109/tc.2022.3223302
• 发表时间: 2023-06
• 期刊: IEEE Transactions on Computers
• 影响因子: 3.7
• 作者: Jiang Zhou;Yong Chen;Mai Zheng;Weiping Wang

Optimal GPU Frequency Selection using Multi-Objective Approaches for HPC Systems

• DOI: 10.1109/hpec55821.2022.9926317
• 发表时间: 2022-09
• 期刊: 2022 IEEE High Performance Extreme Computing Conference (HPEC)
• 作者: Ghazanfar Ali;Sridutt Bhalachandra;N. Wright;Mert Side;Yong Chen

HAM: Hotspot-Aware Manager for Improving Communications With 3D-Stacked Memory

HAM：热点感知管理器，用于改善 3D 堆栈内存的通信

• DOI: 10.1109/tc.2021.3066982
• 发表时间: 2021
• 期刊: IEEE Transactions on Computers
• 影响因子: 3.7
• 作者: Wang, Xi;Tumeo, Antonino;Leidel, John D.;Li, Jie;Chen, Yong
• 通讯作者: Chen, Yong

JobViewer: Graph-based Visualization for Monitoring High-Performance Computing System

JobViewer：用于监控高性能计算系统的基于图形的可视化

• DOI: 10.1109/bdcat56447.2022.00021
• 发表时间: 2022
• 期刊: IEEE
• 作者: Dang, Tommy;Nguyen, Ngan V.T.;Li, Jie;Sill, Alan;Hass, Jon;Chen, Yong
• 通讯作者: Chen, Yong

Exploiting user activeness for data retention in HPC systems

利用用户活跃度来保留 HPC 系统中的数据

• DOI: 10.1145/3458817.3476201
• 发表时间: 2021
• 作者: Zhang, Wei;Byna, Suren;Sim, Hyogi;Lee, Sangkeun;Vazhkudai, Sudharshan;Chen, Yong
• 通讯作者: Chen, Yong