SHF: Small: Methods and Architectures for Optimization and Hardware Acceleration of Spiking Neural Networks

SHF：小型：尖峰神经网络优化和硬件加速的方法和架构

• 批准号: 2310170
• 负责人: Peng Li
• 金额: $ 59.93万
• 依托单位: University of California-Santa Barbara
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-09-15 至 2026-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2310170&HistoricalAwards=false
• 关键词: SHF; Small; Methods; Architectures; Optimization

Artificial intelligence is a powerful cross-cutting technology and is expected to promote broad advancements in science and technology as well as foster social benefits. To this end, exploring novel computational principles inspired by the brain may offer promising new avenues to enable artificial intelligence. This project is positioned to address key challenges in designing and engineering brain-inspired spiking neural models. As such, it may lead to methods, tools, and hardware system designs that will ultimately support new generations of software- and hardware-based artificial intelligence systems with potentially significantly improved performance and efficiency. This project will produce educational materials to be integrated into undergraduate- and graduate-level curricula on artificial intelligence and hardware system design, thereby providing workforce training opportunities in these areas of importance. The principal investigator will actively recruit undergraduate, underrepresented, and female students for research participation and training while partnering with various outreach programs. The results of this award may be derived in a variety of forms, including algorithms, software design tools, and hardware architectures and implementations that will be disseminated in broad research and industrial communities through publications, workshops, talks, and research collaborations. Engagement with US high-tech industries and other research organizations will be sought to broaden the impact of this work, promote potential technology transfer into practice, and offer additional mentoring and training of students under diverse industrial and research settings.Deep learning based on conventional non-spiking artificial neural networks (ANNs) has achieved great success in many application domains in recent years. Nevertheless, the conventional ANNs cannot immediately explore temporal codes and lack energy-efficient event-based processing. On the other hand, it is believed that attaining near-human-level intelligence requires computing paradigms that better mimic biological brains. As such, spiking neural networks (SNNs) offer a complementary biologically-plausible approach to facilitating future artificial intelligence systems. However, there are key roadblocks to a wider adoption of spiking neural networks. SNNs are much harder to train than conventional ANNs. There is a general lack of insights and systemic approaches for designing computationally-powerful SNNs, particularly SNNs with recurrent connections. Hardware acceleration of SNNs is hampered by complex data dependencies across both time and space, and unstructured firing sparsity. This work will start out by developing much needed accurate SNN training methods that can robustly learn precise temporal behavior and jointly tune spike count and spike timing. Scalable architectural design of recurrent SNNs and novel automated spiking neural structural optimization methods will be developed to support the design of computationally powerful SNNs. To enable energy-efficient high-throughput hardware acceleration, dedicated SNN hardware accelerator architectures that minimize expensive data movements and facilitate parallel processing in both space and time will be designed. Application-independent spike coding, spike compression, and architectures exploring unstructured firing sparsity will be investigated for SNN hardware acceleration. High-performance SNN hardware accelerators will be demonstrated on field-programmable gate-array devices.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

人工智能是一项强大的交叉技术，有望促进科学技术的广泛进步并促进社会效益。为此，探索受大脑启发的新颖计算原理可能会为实现人工智能提供有希望的新途径。该项目旨在解决设计和工程受大脑启发的尖峰神经模型的关键挑战。因此，它可能会带来方法、工具和硬件系统设计，最终支持新一代基于软件和硬件的人工智能系统，并可能显着提高性能和效率。该项目将制作教育材料，纳入本科生和研究生水平的人工智能和硬件系统设计课程，从而为这些重要领域提供劳动力培训机会。首席研究员将积极招募本科生、代表性不足的学生和女学生参与研究和培训，同时与各种外展项目合作。该奖项的成果可能会以多种形式产生，包括算法、软件设计工具以及硬件架构和实现，这些成果将通过出版物、研讨会、讲座和研究合作在广泛的研究和工业界传播。将寻求与美国高科技行业和其他研究组织的合作，以扩大这项工作的影响，促进潜在的技术转化为实践，并在不同的工业和研究环境下为学生提供额外的指导和培训。基于传统非传统的深度学习-尖峰人工神经网络（ANN）近年来在许多应用领域取得了巨大成功。然而，传统的人工神经网络无法立即探索时间代码，并且缺乏节能的基于事件的处理。另一方面，人们相信获得接近人类水平的智能需要更好地模仿生物大脑的计算范式。因此，尖峰神经网络（SNN）提供了一种互补的生物学上合理的方法来促进未来的人工智能系统。然而，更广泛地采用尖峰神经网络存在一些关键障碍。 SNN 比传统的 ANN 更难训练。普遍缺乏设计计算能力强大的 SNN 的见解和系统方法，特别是具有循环连接的 SNN。 SNN 的硬件加速受到跨时间和空间的复杂数据依赖性以及非结构化发射稀疏性的阻碍。这项工作将从开发急需的准确 SNN 训练方法开始，这些方法可以稳健地学习精确的时间行为并联合调整尖峰计数和尖峰时序。将开发循环 SNN 的可扩展架构设计和新颖的自动尖峰神经结构优化方法，以支持计算能力强大的 SNN 的设计。为了实现节能的高吞吐量硬件加速，将设计专用的 SNN 硬件加速器架构，以最大限度地减少昂贵的数据移动并促进空间和时间上的并行处理。将研究独立于应用程序的尖峰编码、尖峰压缩和探索非结构化发射稀疏性的架构，以实现 SNN 硬件加速。高性能 SNN 硬件加速器将在现场可编程门阵列设备上进行演示。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力优势和更广泛的影响审查标准进行评估，被认为值得支持。