SHF: Small: Expediting the Execution of Machine Learning Applications on Multi-GPU Infrastructure with Architecture Awareness and Runtime Support

SHF：小型：通过架构意识和运行时支持加快多 GPU 基础设施上机器学习应用程序的执行

• 批准号: 2154973
• 负责人: Xulong Tang
• 金额: $ 59.99万
• 依托单位: University of Pittsburgh
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-06-15 至 2025-05-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2154973&HistoricalAwards=false
• 关键词: SHF; Small; Expediting; Execution; Machine

Deep Neural Networks (DNNs) have become one of the most popular machine-learning techniques for solving real-world problems in object classification, autonomous vehicles, natural language processing, etc. Due to the ever-growing problem size and complexity, the training and inference of DNN models are increasingly time-consuming and require enormous computing resources. As such, multi-GPU infrastructure is a desirable platform that has been widely used in modern DNN tasks. However, the delivered DNN execution scalability is severely limited due to architectural unawareness and lacking easy-to-use runtime support. This research uncovers and addresses the architectural bottlenecks of DNN executions. The outcome of this research is expected to achieve scalable DNN executions on multi-GPU infrastructure. The educational and outreach components of this project include (i) new course projects on multi-GPU infrastructure integrated into graduate-level computer architecture courses; (ii) engaging undergraduate students in the research activities through senior Capstone project courses and an outreach program at PI’s institute; and (iii) increasing the participation and visibility of female and minority students in computer architecture, computer science, and engineering.This research is set to uncover and address the architectural bottlenecks of DNN executions on multi-GPUs. Specifically: 1) It identifies address translation as an essential bottleneck in multi-GPU performance. It redesigns the Translation Lookaside Buffer (TLB) hierarchy and the page table walk for both single-tenant and multi-tenant DNN executions on multi-GPU infrastructure. 2) It investigates the data-movement overheads in data parallelism and model parallelism of modern DNN applications. It proposes architecture-aware data distillation and neuron-based model partitioning to mitigate the data movement overheads. 3) It proposes a runtime framework that fosters the usage of multi-GPUs through enhanced programmability, which allows dynamic and automatic virtual kernel to physical kernel generation during execution.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

深度神经网络（DNN）已成为最受欢迎的机器学习技术之一，用于解决对象分类，自动驾驶汽车，自然语言处理等现实世界中的问题。由于问题的大小和复杂性，DNN模型的培训和推理越来越耗时，并且需要增强的计算资源。因此，多GPU基础架构是一个理想的平台，已被广泛用于现代DNN任务。但是，由于架构不认识和缺乏易于使用的运行时支持，交付的DNN的可执行可伸缩性受到严重限制。这项研究揭示了DNN执行的建筑瓶颈并解决了。这项研究的结果有望在多GPU基础架构上实现可扩展的DNN执行。该项目的教育和外展组成部分包括（i）集成到研究生级计算机架构课程中的多GPU基础设施的新课程项目； （ii）通过高级Capstone项目课程和PI Institute的外展计划，让本科生从事研究活动； （iii）增加女性和少数族裔学生参与计算机架构，计算机科学和工程的参与和可见性。这项研究将揭示和解决多GPU上DNN执行的建筑瓶颈。具体：1）它将地址翻译识别为多GPU性能中的必要瓶颈。它重新设计了lookAside缓冲区（TLB）层次结构的翻译，以及在多GPU基础架构上的单租户和多租户DNN执行的页面表步行。 2）它研究了现代DNN应用的数据并行性和模型并行性中的数据移动开销。它建议架构感知数据蒸馏和基于神经元的模型分区，以减轻数据运动开销。 3）它提出了一个运行时框架，该框架通过增强的程序可培养多GPU的使用，该框架可以在执行过程中动态和自动虚拟内核到物理内核生成。该奖项反映了NSF的法定任务，并通过使用基金会的知识分子优点和更广泛的影响审查标准来评估被认为是宝贵的支持。

项目成果

SmartFRZ: An Efficient Training Framework using Attention-Based Layer Freezing

• DOI: 10.48550/arxiv.2401.16720
• 发表时间: 2024-01
• 期刊: ArXiv
• 作者: Sheng Li;Geng Yuan;Yuezhen Dai;Youtao Zhang;Yanzhi Wang;Xulong Tang

Adversarial Prefetch: New Cross-Core Cache Side Channel Attacks

• DOI: 10.1109/sp46214.2022.9833692
• 发表时间: 2021-10
• 期刊: 2022 IEEE Symposium on Security and Privacy (SP)
• 作者: Yanan Guo;Andrew Zigerelli;Youtao Zhang;Jun Yang

Trans-FW: Short Circuiting Page Table Walk in Multi-GPU Systems via Remote Forwarding

• DOI: 10.1109/hpca56546.2023.10071054
• 发表时间: 2023-02
• 期刊: 2023 IEEE International Symposium on High-Performance Computer Architecture (HPCA)
• 作者: Bingyao Li;Jieming Yin;Anup Holey;Youtao Zhang;Jun Yang;Xulong Tang

CEGMA: Coordinated Elastic Graph Matching Acceleration for Graph Matching Networks

• DOI: 10.1109/hpca56546.2023.10070956
• 发表时间: 2023-02
• 期刊: 2023 IEEE International Symposium on High-Performance Computer Architecture (HPCA)
• 作者: Yuezhen Dai;Youtao Zhang;Xulong Tang

AB-ORAM: Constructing Adjustable Buckets for Space Reduction in Ring ORAM

• DOI: 10.1109/hpca56546.2023.10071064
• 发表时间: 2023-02
• 期刊: 2023 IEEE International Symposium on High-Performance Computer Architecture (HPCA)
• 作者: Mehrnoosh Raoufi;Jun Yang;Xulong Tang;Youtao Zhang