FET: Small: RESONANCE: Accelerating Speech/Language Processing through Collective Training using Commodity ReRAM Chips

FET：小型：共振：使用商用 ReRAM 芯片通过集体训练加速语音/语言处理

• 批准号: 1910299
• 负责人: Hai Li
• 金额: $ 50万
• 依托单位: Duke University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-10-01 至 2023-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1910299&HistoricalAwards=false
• 关键词: FET; Small; RESONANCE; Accelerating; Speech

Moving machine learning techniques from the computing cloud down to edge computing nodes closer to the user is highly desirable in many use cases that require quick responses from the collected data sets. A typical use scenario is multi-task applications where a cloud server retains well-trained large-scale models, which are deployed in edge devices based on specific local needs. Examples include language translation or speech recognition with accents in multi-language audio conferences. However, supporting multi-task application on edge devices is challenging due to the associated high computational cost and large variety of involved models. Very little effort has been spent on the corresponding hardware design, especially for supporting multi-task speech and natural language processing (NLP) applications on edge compute devices. This research aims to design a novel computing system dedicated to such multi-task applications, particularly on accelerating speech/NLP, by combining innovations in both algorithm and hardware domains. The study benefits big data research, and industry at large by inspiring an interactive design philosophy between the designs of speech/NLP algorithms and the corresponding computing platforms. Undergraduate and graduate students involved in this research will be trained for the next-generation information technology workforce. Different from conventional edge computing devices that mainly focuses on balancing the workloads between the cloud and the edge devices and optimizing the communication in between, this project concentrates on how to efficiently decompose and compress the task-specific sub-models extracted from a large multi-task model in the cloud so that deployment of the edge devices meet the functionality and performance needs under the specific hardware constraint. More specifically, the algorithm-level innovations enable a decomposable speech/NLP model that always assures proper function and performance in resource-limited edge devices, while the hardware-level innovations allow these devices to efficiently support speech/NLP multi-task applications and unleash the great potential of Resistive Random Access Memory (ReRAM)-based computing platforms. During the real-time operation, the model on the edge device can be scaled-up or shrunk-down to accommodate the dynamic hardware environment and user needs. The research leads to a holistic methodology across algorithm redesign, hardware acceleration, and an integrated software/hardware co-design.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在许多需要从收集的数据集快速响应的用例中，将机器学习技术从计算云转移到更靠近用户的边缘计算节点是非常可取的。典型的使用场景是多任务应用程序，其中云服务器保留经过良好训练的大规模模型，这些模型根据特定的本地需求部署在边缘设备中。 示例包括多语言音频会议中的语言翻译或带口音的语音识别。然而，由于相关的高计算成本和大量涉及的模型，在边缘设备上支持多任务应用程序具有挑战性。在相应的硬件设计上花费的精力很少，特别是在支持边缘计算设备上的多任务语音和自然语言处理（NLP）应用方面。本研究旨在通过结合算法和硬件领域的创新，设计一种专用于此类多任务应用的新型计算系统，特别是加速语音/NLP。该研究通过激发语音/NLP 算法设计与相应计算平台之间的交互设计理念，使大数据研究和整个行业受益。参与这项研究的本科生和研究生将接受下一代信息技术劳动力的培训。与传统的边缘计算设备主要关注平衡云端和边缘设备之间的工作负载并优化两者之间的通信不同，该项目专注于如何有效地分解和压缩从大型多目标中提取的特定于任务的子模型。云中的任务模型，以便边缘设备的部署满足特定硬件约束下的功能和性能需求。更具体地说，算法级创新实现了可分解的语音/NLP模型，始终确保资源有限的边缘设备的正常功能和性能，而硬件级创新使这些设备能够有效支持语音/NLP多任务应用程序并释放基于电阻式随机存取存储器 (ReRAM) 的计算平台的巨大潜力。在实时操作过程中，边缘设备上的模型可以放大或缩小，以适应动态的硬件环境和用户需求。该研究形成了涵盖算法重新设计、硬件加速和集成软件/硬件协同设计的整体方法。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力优点和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

BSQ: Exploring Bit-Level Sparsity for Mixed-Precision Neural Network Quantization

• 发表时间: 2021-02
• 期刊: ArXiv
• 作者: Huanrui Yang;Lin Duan;Yiran Chen;Hai Li

SpikeSen: Low-Latency In-Sensor-Intelligence Design With Neuromorphic Spiking Neurons

SpikeSen：具有神经形态尖峰神经元的低延迟传感器内智能设计

• DOI: 10.1109/tcsii.2023.3235888
• 发表时间: 2023
• 期刊: IEEE Transactions on Circuits and Systems II: Express Briefs
• 作者: Li, Ziru;Zheng, Qilin;Chen, Yiran;Li, Hai
• 通讯作者: Li, Hai

Processing-in-Memory Technology for Machine Learning: From Basic to ASIC

用于机器学习的内存处理技术：从基础到 ASIC

• DOI: 10.1109/tcsii.2022.3168404
• 发表时间: 2022
• 期刊: IEEE Transactions on Circuits and Systems II: Express Briefs
• 作者: Taylor, Brady;Zheng, Qilin;Li, Ziru;Li, Shiyu;Chen, Yiran
• 通讯作者: Chen, Yiran

ReBoc: Accelerating Block-Circulant Neural Networks in ReRAM

• DOI: 10.23919/date48585.2020.9116422
• 发表时间: 2020-03
• 期刊: 2020 Design, Automation & Test in Europe Conference & Exhibition (DATE)
• 作者: Yitu Wang;Fan Chen;Linghao Song;C. Richard Shi;H. Li;Yiran Chen

Parallelism in Deep Learning Accelerators

• DOI: 10.1109/asp-dac47756.2020.9045206
• 发表时间: 2020-01
• 期刊: 2020 25th Asia and South Pacific Design Automation Conference (ASP-DAC)
• 作者: Linghao Song;Fan Chen;Yiran Chen;H. Li