CAREER: Compiler and Runtime Support for Multi-Tasking on Commodity GPUs

职业：商用 GPU 上多任务的编译器和运行时支持

• 批准号: 1750760
• 负责人: Bo Wu
• 金额: $ 50.15万
• 依托单位: Colorado School of Mines
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-05-01 至 2024-04-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1750760&HistoricalAwards=false
• 关键词: CAREER; Compiler; Runtime; Support; Multi

General-purpose Graphics Processing Units (GPU) computing has become mainstream, as witnessed in various domains such as machine learning, graph analytics, and scientific simulation. One notable trend is employing GPUs in data centers and cloud computing infrastructures to satisfy users' increasing demand to accelerate their applications. In such multi-tasking environments, applications from different users contend to use the shared GPU, leading to unpredictable and unacceptable performance degradation. This CAREER project aims at developing a set of compiler and runtime techniques to support multi-tasking on commodity GPUs in a transparent and efficient manner. The compiler techniques circumvent the hardware limitations to enable a set of features, such as preemption, and the runtime system schedules applications to utilize the potential of the GPU and guarantees quality of service. In addition, the investigator advances GPU education in the University to target both Computer Science (CS) and non-CS students based on a GPU education center.Specifically, the project investigates how to integrate compiler and runtime techniques to support multi-tasking on GPUs by building a system that achieves three goals. First, the system addresses GPU core contention by enabling flexible GPU kernel preemption. The compiler transforms the GPU program to be a preemptable form by circumventing the limitation imposed by the hardware thread scheduler. The runtime intercepts all GPU kernel launch requests and makes global preemption and scheduling decisions to maximize performance. Second, the system supports fine-grained sharing for threads from different applications to fully utilize hardware resources within GPU streaming multi-processors. The runtime guarantees the QoS of user-facing applications while optimizing overall throughput aided by performance prediction. Third, the system addresses GPU memory contention by coordinating GPU memory transfers, which considers memory access patterns and array reuse patterns.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

通用图形处理单元 (GPU) 计算已成为主流，机器学习、图形分析和科学模拟等各个领域都见证了这一点。一个显着的趋势是在数据中心和云计算基础设施中使用 GPU 来满足用户日益增长的加速应用程序的需求。在这种多任务环境中，不同用户的应用程序会争用共享 GPU，从而导致不可预测且不可接受的性能下降。该 CAREER 项目旨在开发一套编译器和运行时技术，以透明且高效的方式支持商用 GPU 上的多任务处理。编译器技术规避了硬件限制，以启用一系列功能，例如抢占，并且运行时系统调度应用程序以利用 GPU 的潜力并保证服务质量。此外，研究人员还基于 GPU 教育中心推进大学的 GPU 教育，以针对计算机科学 (CS) 和非 CS 学生。具体来说，该项目研究如何集成编译器和运行时技术以支持 GPU 上的多任务处理通过建立一个实现三个目标的系统。首先，系统通过启用灵活的 GPU 内核抢占来解决 GPU 内核争用问题。编译器通过规避硬件线程调度器施加的限制，将GPU程序转换为可抢占的形式。运行时拦截所有 GPU 内核启动请求，并做出全局抢占和调度决策，以最大限度地提高性能。其次，系统支持不同应用线程的细粒度共享，充分利用GPU流多处理器内的硬件资源。运行时保证面向用户的应用程序的 QoS，同时通过性能预测优化整体吞吐量。第三，该系统通过协调 GPU 内存传输来解决 GPU 内存争用，其中考虑了内存访问模式和阵列重用模式。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

DGSM: A GPU-Based Subgraph Isomorphism framework with DFS exploration

DGSM：基于 GPU 的子图同构框架，具有 DFS 探索

• DOI: 10.1109/rsdha56811.2022.00006
• 发表时间: 2022-11-01
• 期刊: 2022 IEEE/ACM Redefining Scalability for Diversely Heterogeneous Architectures Workshop (RSDHA)
• 作者: W. Han;Connor Holmes;Bo Wu
• 通讯作者: Bo Wu

GraphZero: A High-Performance Subgraph Matching System

GraphZero：高性能子图匹配系统

• DOI: 10.1145/3469379.3469383
• 发表时间: 2021-06
• 期刊: ACM SIGOPS Operating Systems Review
• 作者: Mawhirter, Daniel;Reinehr, Sam;Holmes, Connor;Liu, Tongping;Wu, Bo
• 通讯作者: Wu, Bo

Dryadic: Flexible and Fast Graph Pattern Matching at Scale

Dryadic：灵活快速的大规模图形模式匹配

• DOI: 10.1109/pact52795.2021.00028
• 发表时间: 2021-01
• 期刊: 30th International Conference on Parallel Architectures and Compilation Techniques
• 作者: Mawhirter, Daniel;Reinehr, Samuel;Han, Wei;Fields, Noah;Claver, Miles;Holmes, Connor;McClurg, Jedidiah;Liu, Tongping;Wu, Bo
• 通讯作者: Wu, Bo

GRNN: Low-Latency and Scalable RNN Inference on GPUs

• DOI: 10.1145/3302424.3303949
• 发表时间: 2019-03-25
• 期刊: Proceedings of the Fourteenth EuroSys Conference 2019
• 作者: Connor Holmes;Daniel Mawhirter;Yuxiong He;Feng Yan;Bo Wu
• 通讯作者: Bo Wu

NxMTransformer: Semi-Structured Sparsification for Natural Language Understanding via ADMM

NxMTransformer：通过 ADMM 进行自然语言理解的半结构化稀疏化

• DOI: 10.48550/arxiv.2206.07422
• 发表时间: 2021-10-28
• 期刊: 2022 IEEE/ACM International Conference On Computer Aided Design (ICCAD)
• 作者: Connor Holmes;Minjia Zhang;Yuxiong He;Bo Wu
• 通讯作者: Bo Wu