SHF: Small: Enabling Efficient Context Switching and Effective Latency Hiding in GPUs

SHF：小：在 GPU 中实现高效的上下文切换和有效的延迟隐藏

• 批准号: 1618509
• 负责人: Huiyang Zhou
• 金额: $ 33万
• 依托单位: North Carolina State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2016
• 资助国家: 美国
• 起止时间: 2016-08-01 至 2019-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1618509&HistoricalAwards=false
• 关键词: SHF; Small; Enabling; Efficient; Context

Graphics processing units (GPUs), initially designed for computer graphics, are becoming widely used for general purpose computing. This project addresses two important challenges in GPU computing. First, it investigates schemes to enable GPUs to be preempted efficiently, which is critical for GPUs to satisfy the quality of service (QOS) requirement in the cloud environment. Second, the project looks into approaches to significantly improve the latency hiding capability of GPUs. This interdisciplinary research has two practical uses, efficient preemption empowering GPUs as truly shared resource and effective latency hiding improving both the GPU performance and energy efficiency. Graduate student advising and industry collaboration are two key aspects of the project.The design philosophy of GPUs is to exploit very high degrees of data-level parallelism (DLP), expressed as thread-level parallelism (TLP), to hide long instruction latency. As a side effect, GPUs feature high amounts of on-chip resources to store the contexts or the architectural states of the large numbers of concurrent threads. The large contexts result in long latency for context switching, which makes it difficult for GPUs to be truly shared in cloud servers. This research project leverages the nature of the single-instruction multiple-thread (SIMT) execution model to drastically reduce and compress the GPU context size. Software and hardware approaches are integrated to enable instruction-level preemption for GPUs to meet the QOS requirements. Fast context switching is also used to switch out stalled threads and switch in new ones such that the otherwise idle computing resources can be utilized to provide much higher latency-hiding capability. It essentially achieves higher TLP on GPUs without enlarging their critical on-chip resources.

图形处理单元 (GPU) 最初是为计算机图形设计的，现在正广泛用于通用计算。该项目解决了 GPU 计算中的两个重要挑战。首先，它研究了使 GPU 能够被有效抢占的方案，这对于 GPU 满足云环境中的服务质量 (QOS) 要求至关重要。其次，该项目研究了显着提高 GPU 延迟隐藏能力的方法。这项跨学科研究有两个实际用途：高效抢占使 GPU 成为真正的共享资源，以及有效的延迟隐藏提高 GPU 性能和能源效率。研究生建议和行业合作是该项目的两个关键方面。GPU 的设计理念是利用非常高程度的数据级并行性 (DLP)，表示为线程级并行性 (TLP)，以隐藏较长的指令延迟。副作用是，GPU 具有大量片上资源来存储大量并发线程的上下文或架构状态。大的上下文导致上下文切换的延迟很长，这使得GPU很难在云服务器中真正共享。该研究项目利用单指令多线程 (SIMT) 执行模型的性质来大幅减少和压缩 GPU 上下文大小。通过软硬件结合的方式，实现GPU指令级抢占，满足QOS要求。快速上下文切换还用于切换出停滞的线程并切换到新的线程，以便可以利用空闲的计算资源来提供更高的延迟隐藏能力。它本质上在 GPU 上实现了更高的 TLP，而无需扩大其关键的片上资源。

项目成果

Scatter-and-Gather Revisited: High-Performance Side-Channel-Resistant AES on GPUs

重新审视分散和聚集：GPU 上的高性能抗侧通道 AES

• DOI: 10.1145/3300053.3319415
• 发表时间: 2019-01
• 期刊: Proceedings of the 12th Workshop on General Purpose Processing Using GPUs
• 作者: Lin, Zhen;Mathur, Utkarsh;Zhou, Huiyang
• 通讯作者: Zhou, Huiyang

Coordinated CTA Combination and Bandwidth Partitioning for GPU Concurrent Kernel Execution

GPU 并发内核执行的协调 CTA 组合和带宽分区

• DOI: 10.1145/3326124
• 发表时间: 2019-06-17
• 期刊: ACM Transactions on Architecture and Code Optimization (TACO)
• 作者: Zhen Lin;Hongwen Dai;Mike Mantor;Huiyang Zhou
• 通讯作者: Huiyang Zhou