Collaborative Research: SHF: Small: Quasi Weightless Neural Networks for Energy-Efficient Machine Learning on the Edge

合作研究：SHF：小型：用于边缘节能机器学习的准失重神经网络

• 批准号: 2326895
• 负责人: Subhasish Mitra
• 金额: $ 15万
• 依托单位: Stanford University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-10-01 至 2026-09-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2326895&HistoricalAwards=false
• 关键词: Collaborative; Research; SHF; Small; Quasi

Deep Neural Networks (DNNs) have recently enabled revolutionary advances in a wide variety of tasks, however these deep networks demand large amounts of memory and computation resources. Such demands can be highly difficult (or even impractical) for systems on the edge. Although DNNs are very accurate, the energy consumed by DNNs is orders of magnitude higher than biological neural activities for similar tasks. It is important to reduce the computational and energy demands of machine learning hardware so that inferencing on the edge can become a low-cost, low-energy task. Weightless Neural Networks (WNNs) represent a distinct class of neural models which derive inspiration from the processing of input signals by the dendritic trees of biological neurons. WNNs do not use weights or multiply-add operations to determine their responses. Instead, they rely on value lookups implemented using look-up tables. This project explores small models that are more energy efficient compared to multiplication-and-addition-based deep learning models. WNNs are very promising from the perspective of energy-efficiency, and low latency, and our effort is directed at enabling a myriad of ultra-low energy edge applications otherwise impossible. This project explores low-energy machine learning hardware which combine the benefits of traditional DNNs and the computation-less weightless neural networks. Techniques used include (1) utilizing multi-layer networks and hierarchical networks to create novel weightless neural network architectures, (2) devising novel training algorithms for WNNs utilizing multi-shot training with feedback (3) exploring quasi-weightless neural networks using emerging novel memory technologies, and (4) designing systems for energy-efficient edge intelligence. The collaborative project between the University of Texas and Stanford University innovates across multiple layers of the system stack, including architecture and circuit layers. The collaborative activity between the University of Texas and Stanford involves many underrepresented communities from a STEM perspective, including minority and women, undergrads, and first-generation college students.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

深度神经网络（DNNS）最近在各种任务中实现了革命性的进步，但是这些深层网络需要大量的内存和计算资源。对于边缘的系统而言，这种需求可能非常困难（甚至是不切实际）。尽管DNN非常准确，但DNN消耗的能量的数量级比生物神经活动高的数量级。 重要的是减少机器学习硬件的计算和能源需求，以便在边缘上推断可以成为一项低成本，低能的任务。失重的神经网络（WNN）代表了一类独特的神经模型，这些神经模型从生物神经元的树突树的处理中得出了灵感。 WNN不使用权重或乘以ADD操作来确定其响应。相反，他们依靠使用查找表实现的价值查找。 该项目探讨了与基于乘法和基础深度学习模型相比，这些模型更节能。从能源效率的角度来看，WNN非常有前途，并且潜伏期低，我们的努力旨在使无数的超低能量边缘应用否则不可能。该项目探索了低能机器学习硬件，该硬件结合了传统DNN的好处和无计算的失重神经网络。 所使用的技术包括（1）利用多层网络和分层网络来创建新型的失重神经网络体系结构，（2）设计新型培训算法，用于WNNS，利用反馈（3）探索Quasi-Weightss weightnextless神经网络，使用新颖的新型记忆技术和（4）设计能源（4）设计效果。德克萨斯大学和斯坦福大学之间的合作项目跨系统堆栈的多层创新，包括建筑和电路层。得克萨斯大学和斯坦福大学之间的合作活动涉及许多人代表性不足的社区，包括少数群体和妇女，本科生和第一代大学生。该奖项反映了NSF的法定任务，并被认为是通过基金会的知识分子和宽广的影响来评估的支持，并被认为是值得的。