CAREER: Efficient Large Language Model Inference Through Codesign: Adaptable Software Partitioning and FPGA-based Distributed Hardware

职业：通过协同设计进行高效的大型语言模型推理：适应性软件分区和基于 FPGA 的分布式硬件

• 批准号: 2339084
• 负责人: Mohamed Abdelfattah
• 金额: $ 88.31万
• 依托单位: Cornell University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2024
• 资助国家: 美国
• 起止时间: 2024-05-01 至 2029-04-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2339084&HistoricalAwards=false
• 关键词: CAREER; Efficient; Large; Language; Model

Artificial intelligence (AI) has entered the "age of scale". Huge amounts of training data are being used to train enormous deep neural networks (DNNs) on large-scale computers as epitomized by the rise of large language models (LLMs). The extremely high demand for this technology is clearly evident, as recently exemplified by ChatGPT: an LLM chatbot that garnered 100 million active users merely two months post-release, setting a new world record. However, deploying LLMs can be quite costly, given that their memory footprint can extend to terabytes of data while also demanding high computational resources. Consequently, large-scale distributed computers have become essential, particularly to meet the performance required for interactive applications. To improve efficiency, this project tackles new challenges that are specific to LLMs, including their large memory footprint, varying computational demands, and distributed computing. This is critical to make LLMs more accessible and sustainable for widespread use. Concurrently, this award seeks to develop a diverse AI workforce proficient in algorithms, hardware, and software, achieved through a large-scale AI course for diverse student population at public universities, comprehensive curriculum integration, and student mentorship at both graduate and undergraduate levels.This project will enable the codesign of LLMs and distributed computing platforms, divided into three major thrusts that correspond to three levels of the computing stack: software, hardware, and algorithms. Initially, the project will focus on automated partitioning and mapping algorithms, as these form the foundations by which LLMs can be deployed and optimized on both existing and new distributed computing platforms. Key to this research thrust is the development of an extensible hardware performance estimator that can model current GPU-based systems alongside new distributed computing approaches. In particular, the second thrust investigates the use of in-network and near-storage FPGAs within distributed systems to speed up LLM inference. The final thrust investigates platform-aware compression for LLMs, including mixed-precision quantization and low-rank approximation. In addition to improving LLM efficiency across the computing stack, this project will develop a research framework to synergistically co-optimize LLMs and distributed hardware platforms, resulting in new optimized LLM computing systems and implementation methodologies.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

人工智能（AI）已进入“规模时代”。大量的训练数据被用来在大型计算机上训练巨大的深度神经网络（DNN），大型语言模型（LLM）的兴起就是一个缩影。对这项技术的极高需求是显而易见的，最近的 ChatGPT 就是一个例子：一款 LLM 聊天机器人在发布仅两个月后就获得了 1 亿活跃用户，创下了新的世界纪录。然而，部署 LLM 的成本可能相当高，因为它们的内存占用可能会扩展到 TB 级的数据，同时还需要大量的计算资源。因此，大规模分布式计算机变得至关重要，特别是为了满足交互式应用程序所需的性能。为了提高效率，该项目解决了法学硕士特有的新挑战，包括大内存占用、不同的计算需求和分布式计算。这对于使法学硕士更容易获得和可持续广泛使用至关重要。同时，该奖项旨在通过针对公立大学多元化学生群体的大规模人工智能课程、全面的课程整合以及研究生和本科生阶段的学生指导来培养精通算法、硬件和软件的多元化人工智能劳动力。该项目将实现法学硕士和分布式计算平台的协同设计，分为三个主要方向，对应计算堆栈的三个级别：软件、硬件和算法。最初，该项目将重点关注自动分区和映射算法，因为这些算法构成了 LLM 在现有和新的分布式计算平台上部署和优化的基础。这项研究的关键是开发可扩展的硬件性能估计器，该估计器可以对当前基于 GPU 的系统以及新的分布式计算方法进行建模。特别是，第二个重点研究了分布式系统中网络内和近存储 FPGA 的使用，以加速 LLM 推理。最后的重点是研究 LLM 的平台感知压缩，包括混合精度量化和低秩近似。除了提高整个计算堆栈的 LLM 效率外，该项目还将开发一个研究框架，以协同优化 LLM 和分布式硬件平台，从而产生新的优化的 LLM 计算系统和实施方法。该奖项反映了 NSF 的法定使命，并被视为值得通过使用基金会的智力优点和更广泛的影响审查标准进行评估来支持。