SHF: Small: ML Accelerator Cohort Architecture

SHF：小型：ML 加速器群组架构

• 批准号: 2224319
• 负责人: Murali Annavaram
• 金额: $ 60万
• 依托单位: University of Southern California
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-10-01 至 2025-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2224319&HistoricalAwards=false
• 关键词: SHF; Small; ML; Accelerator; Cohort

Machine learning (ML) models play an increasingly crucial role in people's daily lives: from autonomous driving, health care, data center management, to machine translation. To deal with the range of usages, model diversity has been growing: from convolutional neural networks, to embedding table-based recommendation systems, to graph neural networks to natural language processing. As model usage embeds in sensitive domains, such as speech- and emotion-recognition engines, data privacy is also a key requirement for the next generation ML hardware. As the model diversity has grown, the compute and communication demands also vary widely, both across models and within a model. And new privacy protecting execution models such as multi-party computing (MPC) demand fundamentally different hardware support. Hence, there is a need to rethink the design of ML hardware accelerators for the new era of privacy-preserving heterogeneous model usage. To address these concerns, this project focuses on the development of the ML Accelerator Cohort Architecture (MLACA). MLACA is collection of heterogeneous ML accelerator tiles that work in unison to adapt to the dynamically changing ML execution demands. MLACA uses multiple research thrusts to achieve its goals: the first thrust focuses on building a heterogeneous MLACA compute fabric that supports a wide range of dense and sparse execution paradigms, including novel support for private computing. The second thrust focuses on MLACA's memory and acceleration fabric that performs in-memory indexing acceleration for embedding tables, prefetching that uses perfect future knowledge of training data. MLACA's communication thrust focuses on distributed training acceleration, using techniques such as dynamic tensor decomposition that tradeoff computation and communication costs. The runtime system thrust manages MLACA fabric allocation across competing ML models to maximize the resource utilization and improve power efficiency. Technology transition is planned through strong industry collaborations with the USC-Meta center and Intel's Private AI Institute. This research uses NSF’s research experience for undergraduate funding, and USC's internal SURE and K-12 SHINE programs to engage high school students and teachers, and undergraduate students in preparing them for a career in ML systems design.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

机器学习 (ML) 模型在人们的日常生活中发挥着越来越重要的作用：从自动驾驶、医疗保健、数据中心管理到机器翻译，为了应对各种用途，模型的多样性不断增长：从卷积神经网络、从嵌入基于表格的推荐系统，到将神经网络图形化到自然语言处理，随着模型使用嵌入到语音和情感识别引擎等敏感领域，数据隐私也是下一代机器学习硬件的关键要求。模型多样性随着模型的不断增长，计算和通信需求也存在很大差异，无论是在模型之间还是在模型内，新的隐私保护执行模型（例如多方计算（MPC））需要根本不同的硬件支持，因此有必要重新考虑。为隐私保护异构模型使用的新时代设计 ML 硬件加速器 为了解决这些问题，该项目重点开发 ML 加速器队列架构 (MLACA)，它是异构 ML 加速器块的集合。协同工作以适应动态变化的 ML 执行需求 MLACA 使用多个研究重点来实现其目标：第一个重点是构建支持各种密集和稀疏执行范例的异构 MLACA 计算结构，包括新颖的支持。第二个重点是 MLACA 的内存和加速结构，该结构对嵌入表执行内存索引加速，并使用 MLACA 通信的完美未来知识进行预取。 Thrust 专注于分布式训练加速，使用动态张量分解等技术来权衡计算和通信成本。运行时系统 Thrust 管理跨竞争 ML 模型的 MLACA 分配结构，以最大限度地提高资源利用率并提高能源效率。该研究与 USC-Meta 中心和英特尔私人人工智能研究所合作，利用 NSF 的本科生资助研究经验以及 USC 的内部 SURE 和 K-12 SHINE 项目吸引高中生和教师以及本科生参与。为他们从事机器学习系统设计职业做好准备。该奖项反映了 NSF 的法定使命，并且通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

LAORAM: A Look Ahead ORAM Architecture for Training Large Embedding Tables

• DOI: 10.1145/3579371.3589111
• 发表时间: 2021-07
• 期刊: Proceedings of the 50th Annual International Symposium on Computer Architecture
• 作者: Rachit Rajat;Yongqin Wang;M. Annavaram

Characterization of MPC-based Private Inference for Transformer-based Models

• DOI: 10.1109/ispass55109.2022.00025
• 发表时间: 2022-05
• 期刊: 2022 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS)
• 作者: Yongqin Wang;G. Suh;Wenjie Xiong;Benjamin Lefaudeux;Brian Knott;M. Annavaram;Hsien-Hsin S. Lee