CSR: Small: A Fine-Grained Hierarchical Memory Management System for Applications with Dynamic Memory Demand on GPUs

CSR：小型：针对 GPU 上具有动态内存需求的应用程序的细粒度分层内存管理系统

• 批准号: 2311610
• 负责人: Peng Jiang
• 金额: $ 52万
• 依托单位: University of Iowa
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-10-01 至 2026-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2311610&HistoricalAwards=false
• 关键词: CSR; Small; Fine; Grained; Hierarchical

Graphics Processing Units (GPUs) have become an important component of modern high end computer platforms due to their ability to deliver high throughput, efficient computation. GPU utilization is therefore crucial to the performance of many applications. Despite over a decade of study in GPU programming, an important class of applications with dynamic memory demand is not well supported. This research aims to simplify the programming and improve the performance of dynamic-memory applications on GPUs. The outcome of this project will unlock the power of GPUs for a wide variety of applications with complex and dynamic memory usage, including bioinformatics, scientific computing, and machine learning. Additionally, this project will contribute to the development of high-performance computing courses and provide research opportunities to underrepresented students at the institution.There are mainly three challenges for supporting dynamic-memory applications on GPUs. First, the dynamic data sizes in these applications require data storage at multiple memory levels. Accessing and synchronizing data across the memory hierarchy can be complicated and time-consuming. Second, the dynamic memory consumption makes memory allocation nontrivial, as the amount of memory allocated to each data object at different memory levels can significantly affect performance. Third, many of these applications have complex and skewed memory access patterns, which makes it difficult to determine the optimal data placement in the memory hierarchy. This project aims to address the above challenges by introducing a fine-grained, hierarchical memory management system on GPU. The system provides a novel programming interface for managing multi-dimensional tensors in the GPU memory hierarchy, which will facilitate the development of dynamic-memory applications. Our system also provides automatic memory pre-allocation and memory access optimizations, which simplify performance optimization for these applications.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

图形处理单元 (GPU) 因其提供高吞吐量、高效计算的能力而成为现代高端计算机平台的重要组成部分。因此，GPU 利用率对于许多应用程序的性能至关重要。尽管对 GPU 编程进行了十多年的研究，但具有动态内存需求的一类重要应用程序并未得到很好的支持。这项研究旨在简化编程并提高 GPU 上动态内存应用的性能。该项目的成果将释放 GPU 的强大功能，适用于具有复杂和动态内存使用的各种应用，包括生物信息学、科学计算和机器学习。此外，该项目将有助于高性能计算课程的开发，并为该机构中代表性不足的学生提供研究机会。支持 GPU 上的动态内存应用主要面临三个挑战。首先，这些应用程序中的动态数据大小需要多个内存级别的数据存储。跨内存层次结构访问和同步数据可能非常复杂且耗时。其次，动态内存消耗使得内存分配变得非常重要，因为分配给不同内存级别的每个数据对象的内存量会显着影响性能。第三，许多此类应用程序具有复杂且扭曲的内存访问模式，这使得很难确定内存层次结构中的最佳数据放置。该项目旨在通过在 GPU 上引入细粒度、分层内存管理系统来解决上述挑战。该系统提供了一种新颖的编程接口，用于管理 GPU 内存层次结构中的多维张量，这将有助于动态内存应用程序的开发。我们的系统还提供自动内存预分配和内存访问优化，从而简化了这些应用程序的性能优化。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力优点和更广泛的影响审查标准进行评估，被认为值得支持。