Collaborative Research: SHF: Small: Quasi Weightless Neural Networks for Energy-Efficient Machine Learning on the Edge

合作研究：SHF：小型：用于边缘节能机器学习的准失重神经网络

• 批准号: 2326894
• 负责人: Lizy John
• 金额: $ 45万
• 依托单位: University of Texas at Austin
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-10-01 至 2026-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2326894&HistoricalAwards=false
• 关键词: Collaborative; Research; SHF; Small; Quasi

Deep Neural Networks (DNNs) have recently enabled revolutionary advances in a wide variety of tasks, however these deep networks demand large amounts of memory and computation resources. Such demands can be highly difficult (or even impractical) for systems on the edge. Although DNNs are very accurate, the energy consumed by DNNs is orders of magnitude higher than biological neural activities for similar tasks. It is important to reduce the computational and energy demands of machine learning hardware so that inferencing on the edge can become a low-cost, low-energy task. Weightless Neural Networks (WNNs) represent a distinct class of neural models which derive inspiration from the processing of input signals by the dendritic trees of biological neurons. WNNs do not use weights or multiply-add operations to determine their responses. Instead, they rely on value lookups implemented using look-up tables. This project explores small models that are more energy efficient compared to multiplication-and-addition-based deep learning models. WNNs are very promising from the perspective of energy-efficiency, and low latency, and our effort is directed at enabling a myriad of ultra-low energy edge applications otherwise impossible. This project explores low-energy machine learning hardware which combine the benefits of traditional DNNs and the computation-less weightless neural networks. Techniques used include (1) utilizing multi-layer networks and hierarchical networks to create novel weightless neural network architectures, (2) devising novel training algorithms for WNNs utilizing multi-shot training with feedback (3) exploring quasi-weightless neural networks using emerging novel memory technologies, and (4) designing systems for energy-efficient edge intelligence. The collaborative project between the University of Texas and Stanford University innovates across multiple layers of the system stack, including architecture and circuit layers. The collaborative activity between the University of Texas and Stanford involves many underrepresented communities from a STEM perspective, including minority and women, undergrads, and first-generation college students.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

深度神经网络 (DNN) 最近在各种任务中取得了革命性的进步，但这些深度网络需要大量的内存和计算资源。对于边缘系统来说，这样的要求可能非常困难（甚至不切实际）。尽管 DNN 非常准确，但 DNN 消耗的能量比类似任务的生物神经活动高出几个数量级。 减少机器学习硬件的计算和能源需求非常重要，这样边缘推理才能成为一项低成本、低能耗的任务。失重神经网络 (WNN) 代表了一类独特的神经模型，其灵感来自生物神经元树突树对输入信号的处理。 WNN 不使用权重或乘加运算来确定其响应。相反，它们依赖于使用查找表实现的值查找。 该项目探索与基于乘法和加法的深度学习模型相比更节能的小型模型。从能源效率和低延迟的角度来看，WNN 非常有前途，我们的努力旨在实现无数超低能耗边缘应用，否则这是不可能的。该项目探索低能耗机器学习硬件，结合了传统 DNN 和无需计算的失重神经网络的优点。 使用的技术包括（1）利用多层网络和分层网络创建新颖的失重神经网络架构，（2）利用带反馈的多镜头训练为 WNN 设计新颖的训练算法（3）利用新兴新颖的方法探索准失重神经网络存储器技术，(4) 设计节能边缘智能系统。德克萨斯大学和斯坦福大学之间的合作项目在系统堆栈的多个层上进行创新，包括架构和电路层。德克萨斯大学和斯坦福大学之间的合作活动涉及许多从 STEM 角度来看代表性不足的社区，包括少数族裔和女性、本科生和第一代大学生。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准。