CAREER: Dynamic Distributed Learning in Spiking Neural Networks with Neural Architecture Search

职业：具有神经架构搜索的尖峰神经网络中的动态分布式学习

• 批准号: 2238227
• 负责人: Priyadarshini Panda
• 金额: $ 50.48万
• 依托单位: Yale University
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-01-01 至 2027-12-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2238227&HistoricalAwards=false
• 关键词: CAREER; Dynamic; Distributed; Learning; Spiking

Artificial Intelligence (AI) has enabled a plethora of applications today, ranging from the most recent chatbots that give you a human-like question/answer experience to autonomous driving cars. But, all these massive feats with AI incur huge costs in terms of energy, memory, and power consumption. In the past decade, Spiking Neural Networks (SNNs) have emerged as a low-power alternative to AI. SNN’s main attraction lies in the fact that they offer low-power architectural implementations, especially for arithmetic operations. Furthermore, unlike traditional neural networks, SNNs process information over time and the temporal dimension, if leveraged suitably, can help enable the next generation of AI applications at lower cost with better performance and robustness. However, training SNNs suitably for realistic tasks has been a long-standing challenge. This project innovates on fundamental optimization strategies, using the temporal features in SNNs to yield new architectures with diverse connectivity and sparsity that yield significant energy-efficiency benefits for distributed low-power edge computing applications. Furthermore, this research will support the interdisciplinary development of a diverse cohort of Ph.D. and undergraduate students and provides a unique education infrastructure to train the next generation of electrical and computer engineering researchers and practitioners.Today, deploying large-scale spiking neural networks (SNNs) for realistic computer vision and related tasks is a non-trivial challenge. This project targets two directions to build large-scale SNNs: 1) We innovate on Neural Architecture Search (NAS) to yield new SNN architectures with temporal feedback connections (that is in stark contrast to conventional feedforward deep learning networks). 2) We use the SNN-specific NAS optimization to perform distributed learning on multiple agents for vision tasks and demonstrate the benefits of using SNNs for low-power edge computing. Particularly, we develop a zero-shot approach that does not require training to search for the optimal network architecture while leveraging temporal and spatial sparsity with pruning and related techniques. This strategy is expected to shorten the design cycle of SNN architecture search by one to two orders of magnitude over existing work. The proposed NAS search will be integrated into a federated learning framework where multiple devices with different resources and data heterogeneity are learning together. Essentially, this project’s framework for discovering new SNN architectures can yield powerful and radical solutions for learning on multiple devices with extreme resource limitations to enable numerous distributed AI applications.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

如今，人工智能 (AI) 已经实现了大量应用，从最新的聊天机器人（为您提供类似人类的问答体验）到自动驾驶汽车，但是，所有这些人工智能的巨大壮举都会带来巨大的能源成本。在过去的十年中，尖峰神经网络 (SNN) 已成为 AI 的低功耗替代品，其主要吸引力在于它们提供低功耗架构实现，尤其是对于人工智能而言。此外，与传统的神经网络不同，SNN 会随着时间和时间维度处理信息，如果利用得当，可以帮助以更低的成本实现下一代人工智能应用，并提供更好的性能和鲁棒性。但是，针对实际任务进行适当的训练。该项目在基本优化策略上进行创新，利用 SNN 中的时间特征来产生具有不同连接性和稀疏性的新架构，从而为分布式低功耗边缘计算应用带来显着的能效优势。此外，这项研究将支持多元化的博士和本科生群体的跨学科发展，并提供独特的教育基础设施来培训下一代电气和计算机工程研究人员和从业人员。用于现实计算机视觉和相关任务的 SNN 是一个不小的挑战，该项目的目标是构建大规模 SNN：1）我们在神经架构搜索（NAS）上进行创新，以产生具有时间反馈连接的新 SNN 架构（那是2）我们使用 SNN 特定的 NAS 优化对视觉任务的多个代理执行分布式学习，并展示使用 SNN 进行低功耗边缘计算的好处，特别是，我们开发了一种不需要训练即可搜索的零样本方法。该策略有望将 SNN 架构搜索的设计周期比现有工作缩短一到两个数量级。提出的 NAS 搜索将被集成到一个联合学习框架中，其中具有不同资源和数据异构性的多个设备一起学习。本质上，该项目用于发现新 SNN 架构的框架可以为在资源极度有限的多个设备上进行学习提供强大而激进的解决方案。支持众多分布式人工智能应用。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。