Collaborative Research: Frameworks: Diamond: Democratizing Large Neural Network Model Training for Science

合作研究：框架：钻石：科学大型神经网络模型训练的民主化

• 批准号: 2401245
• 负责人: Zhao Zhang
• 金额: $ 94.95万
• 依托单位: Rutgers University New Brunswick
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-11-01 至 2026-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2401245&HistoricalAwards=false
• 关键词: Collaborative; Research; Frameworks; Diamond; Democratizing

Diamond is a service designed to democratize access to cutting-edge DL methods by abstracting the use of HPC resources. Diamond combines novel computer science research with translational computer science to reduce the significant barriers that impede adoption of DL methods in science. With Diamond, domain scientists can focus on the neural network architecture design to solve their domain-specific challenges without worrying about Cyberinfrastructure management. Diamond also contributes to key educational outcomes. PhD students work directly on project goals, and tools developed in the project will be used in undergraduate and graduate-level courses. The tools will also be used in summer schools and programs at TACC, UChicago, and NCSA. Targeted recruitment of students from underserved communities at the graduate, undergraduate, and high-school levels will address diversity and outreach goals.Diamond builds upon prior work in software ecosystem management, parallel computing, deep learning, and data management, combining disparate capabilities into a cohesive and user-friendly framework. It provides a web service-enabled programming interface supporting the DL lifecycle from development to deployment and dissemination. It offers container configuration, automatic scaling for distributed training, hyper-parameter tuning, and model sharing. It also applies crucial performance optimizations, including planning for long training jobs, performance-aware model placement, cross-cluster training, and data management. Diamond results are made available to domain scientists, computer scientists, and engineers supporting DL applications in HPC centers.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Diamond 是一项服务，旨在通过抽象 HPC 资源的使用来民主化对尖端 DL 方法的访问。 Diamond 将新颖的计算机科学研究与转化计算机科学相结合，以减少阻碍 DL 方法在科学中采用的重大障碍。借助 Diamond，领域科学家可以专注于神经网络架构设计，以解决特定领域的挑战，而无需担心网络基础设施管理。钻石还为关键的教育成果做出了贡献。博士生直接致力于项目目标，项目中开发的工具将用于本科和研究生课程。这些工具还将用于 TACC、芝加哥大学和 NCSA 的暑期学校和项目。在研究生、本科生和高中阶段有针对性地从服务不足的社区招收学生，将解决多样性和外展目标。Diamond 建立在软件生态系统管理、并行计算、深度学习和数据管理方面的先前工作的基础上，将不同的能力结合成一个具有凝聚力且用户友好的框架。它提供了一个支持 Web 服务的编程接口，支持从开发到部署和传播的 DL 生命周期。它提供容器配置、分布式训练的自动扩展、超参数调整和模型共享。它还应用了关键的性能优化，包括规划长期训练作业、性能感知模型放置、跨集群训练和数据管理。钻石级成果可供支持 HPC 中心 DL 应用的领域科学家、计算机科学家和工程师使用。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Mirage: Towards Low-interruption Services on Batch GPU Clusters with Reinforcement Learning

Mirage：通过强化学习实现批量 GPU 集群上的低中断服务

• DOI: 10.1145/3581784.3607042
• 发表时间: 2023-11
• 期刊: ACM
• 作者: Ding, Qiyang;Zheng, Pengfei;Kudari, Shreyas;Venkataraman, Shivaram;Zhang, Zhao
• 通讯作者: Zhang, Zhao