SHF: Small: ML Accelerator Cohort Architecture

SHF：小型：ML 加速器群组架构

• 批准号: 2224319
• 负责人: Murali Annavaram
• 金额: $ 60万
• 依托单位: University of Southern California
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-10-01 至 2025-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2224319&HistoricalAwards=false
• 关键词: SHF; Small; ML; Accelerator; Cohort

Machine learning (ML) models play an increasingly crucial role in people's daily lives: from autonomous driving, health care, data center management, to machine translation. To deal with the range of usages, model diversity has been growing: from convolutional neural networks, to embedding table-based recommendation systems, to graph neural networks to natural language processing. As model usage embeds in sensitive domains, such as speech- and emotion-recognition engines, data privacy is also a key requirement for the next generation ML hardware. As the model diversity has grown, the compute and communication demands also vary widely, both across models and within a model. And new privacy protecting execution models such as multi-party computing (MPC) demand fundamentally different hardware support. Hence, there is a need to rethink the design of ML hardware accelerators for the new era of privacy-preserving heterogeneous model usage. To address these concerns, this project focuses on the development of the ML Accelerator Cohort Architecture (MLACA). MLACA is collection of heterogeneous ML accelerator tiles that work in unison to adapt to the dynamically changing ML execution demands. MLACA uses multiple research thrusts to achieve its goals: the first thrust focuses on building a heterogeneous MLACA compute fabric that supports a wide range of dense and sparse execution paradigms, including novel support for private computing. The second thrust focuses on MLACA's memory and acceleration fabric that performs in-memory indexing acceleration for embedding tables, prefetching that uses perfect future knowledge of training data. MLACA's communication thrust focuses on distributed training acceleration, using techniques such as dynamic tensor decomposition that tradeoff computation and communication costs. The runtime system thrust manages MLACA fabric allocation across competing ML models to maximize the resource utilization and improve power efficiency. Technology transition is planned through strong industry collaborations with the USC-Meta center and Intel's Private AI Institute. This research uses NSF’s research experience for undergraduate funding, and USC's internal SURE and K-12 SHINE programs to engage high school students and teachers, and undergraduate students in preparing them for a career in ML systems design.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

机器学习（ML）模型在人们的日常生活中起着越来越重要的作用：从自主驾驶，医疗保健，数据中心管理到机器翻译。为了处理使用范围，模型多样性一直在增长：从卷积神经元网络到嵌入基于表的建议系统，再到将神经元网络绘制到自然语言处理。随着嵌入在敏感领域（例如语音和情感识别引擎）中的模型使用情况，数据隐私也是下一代ML硬件的关键要求。随着模型多样性的增长，跨模型和模型中的计算和通信需求也差异很大。以及保护执行模型（例如多方计算（MPC））的新隐私要求从根本上要求不同的硬件支持。因此，有必要重新考虑ML硬件加速器的设计，以实现隐私保护的新时代的异质模型使用。为了解决这些问题，该项目着重于ML加速器队列体系结构（MLACA）的开发。 MLACA是一致工作的异质ML加速器瓷砖的收集，以适应动态变化的ML执行要求。 MLACA使用多个研究推力来实现其目标：第一个推力重点是构建异质的MLACA计算织物，该织物支持广泛的密集和稀疏执行范式，包括对私人计算的新支持。第二个推力着重于MLACA的内存和加速织物，该结构执行内存索引加速度以嵌入表格，预购使用培训数据的完美未来知识。 MLACA的沟通推力专注于分布式培训加速，使用诸如权衡计算和通信成本的动态张量分解等技术。运行时系统推力管理跨竞争ML模型的MLACA织物分配，以最大程度地利用资源利用并提高功率效率。技术过渡是通过与USC-META中心和英特尔的私人AI研究所的强大行业合作计划的。这项研究利用NSF的研究经验用于本科资金，以及USC的内部确保和K-12 Shine计划，以吸引高中生和老师，并在本科生中为他们准备ML Systems Design的职业。这一奖项反映了NSF的法定任务，并通过使用该基金会的知识优点和广泛的影响来评估NSF的法定任务。

项目成果

LAORAM: A Look Ahead ORAM Architecture for Training Large Embedding Tables

• DOI: 10.1145/3579371.3589111
• 发表时间: 2021-07
• 期刊: Proceedings of the 50th Annual International Symposium on Computer Architecture
• 作者: Rachit Rajat;Yongqin Wang;M. Annavaram

Characterization of MPC-based Private Inference for Transformer-based Models

• DOI: 10.1109/ispass55109.2022.00025
• 发表时间: 2022-05
• 期刊: 2022 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS)
• 作者: Yongqin Wang;G. Suh;Wenjie Xiong;Benjamin Lefaudeux;Brian Knott;M. Annavaram;Hsien-Hsin S. Lee