CSR: Small: Collaborative Research: Tuning Extreme-scale Storage Stack through Deep Reinforcement Learning

CSR：小型：协作研究：通过深度强化学习调整超大规模存储堆栈

基本信息

批准号：
1817094
负责人：
Yong Chen
金额：
$ 24万
依托单位：
Texas Tech University
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2018
资助国家：
美国
起止时间：
2018-10-01 至 2024-09-30
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1817094&HistoricalAwards=false
关键词：
CSR Small Collaborative Research Tuning

项目摘要

Many research domains, such as high-energy physics, climate science, astrophysics, combustion science, and computational biology, need to process large amounts of data. Such domains are heavily relying on the capabilities of high performance computing (HPC) systems to manage and efficiently process massive amounts of data. Consequently, applications in the aforementioned research domains require highly optimized performance on the HPC storage systems that store, manage, and manipulate data. This project aims to utilize deep reinforcement learning methods to fine-tune the HPC storage system for optimized performance.This research explores the feasibility of leveraging deep reinforcement learning to optimize HPC storage systems by: (a) Creating a deep learning based HPC storage stack model; (b) Remodeling existing HPC storage stack to support automated configuration and tuning; (c) Collecting training datasets and training the storage stack model; and (d) utilizing the model as a responsive and playable virtual environment to learn the best policy to tune parameters. As a collaborative project, this research aims to advance the domain knowledge of both HPC storage systems and machine learning. The enhanced performance on the HPC storage stack will in turn benefit scientific discovery and thus our society. The investigators will integrate research, education, and outreach efforts during the course of this project, including recruiting and retaining of underrepresented students, mentoring graduate and undergraduate students, integrating research findings into curriculum, and publishing and disseminating results.The data collected to train the storage stack model will be shared at https://discl.cs.ttu.edu/tuningstorage while the code of machine learning at https://github.com/forrestbao/DL4SC. Results and data will be made available by the time of publication. The data will be annotated as appropriate to facilitate interpretation. The principal investigators will strive to maintain the repositories as long as possible.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

许多研究领域，例如高能物理、气候科学、天体物理学、燃烧科学和计算生物学，都需要处理大量数据。这些领域严重依赖高性能计算 (HPC) 系统的功能来管理和高效处理大量数据。因此，上述研究领域的应用程序需要存储、管理和操作数据的 HPC 存储系统具有高度优化的性能。该项目旨在利用深度强化学习方法来微调 HPC 存储系统以优化性能。本研究通过以下方式探索利用深度强化学习来优化 HPC 存储系统的可行性： (a) 创建基于深度学习的 HPC 存储堆栈模型; (b) 改造现有 HPC 存储堆栈以支持自动配置和调整； (c) 收集训练数据集并训练存储堆栈模型； (d) 利用该模型作为响应式且可玩的虚拟环境来学习调整参数的最佳策略。作为一个合作项目，这项研究旨在推进 HPC 存储系统和机器学习的领域知识。 HPC 存储堆栈性能的增强将反过来有利于科学发现，从而有利于我们的社会。研究人员将在该项目过程中整合研究、教育和推广工作，包括招募和保留代表性不足的学生、指导研究生和本科生、将研究成果纳入课程以及发布和传播结果。收集的数据用于培训存储堆栈模型将在 https://discl.cs.ttu.edu/tuningstorage 共享，而机器学习的代码将在 https://github.com/forrestbao/DL4SC 共享。结果和数据将在发布时提供。数据将被适当注释以方便解释。主要研究人员将努力尽可能长时间地维护存储库。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

期刊论文数量（12）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

HiperView: real-time monitoring of dynamic behaviors of high-performance computing centers

HiperView：实时监控高性能计算中心的动态行为

DOI：
10.1007/s11227-021-03724-5
发表时间：
2021-03-29
期刊：
The Journal of Supercomputing
影响因子：
0
作者：
Tommy Dang;Ngan V. T. Nguyen;Yong Chen
通讯作者：
Yong Chen

Automating CPU Dynamic Thermal Control for High Performance Computing

自动化 CPU 动态热控制以实现高性能计算

DOI：
10.1109/ccgrid54584.2022.00061
发表时间：
2022-05-01
期刊：
2022 22nd IEEE International Symposium on Cluster, Cloud and Internet Computing (CCGrid)
影响因子：
0
作者：
Ghazanfar Ali;Lowell Wofford;Christopher Turner;Yong Chen
通讯作者：
Yong Chen

Hashing Based Data Distribution in Heterogeneous Storage

异构存储中基于哈希的数据分布