Collaborative Research: Two-dimensional Synaptic Array for Advanced Hardware Acceleration of Deep Neural Networks

合作研究：用于深度神经网络高级硬件加速的二维突触阵列

• 批准号: 1955246
• 负责人: Yiran Chen
• 金额: $ 20万
• 依托单位: Duke University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-09-01 至 2024-08-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1955246&HistoricalAwards=false
• 关键词: Collaborative; Research; Two; dimensional; Synaptic

Nontechnical:The big data revolution has created a critical need for new computing paradigms to efficiently extract valuable information from large datasets. In existing computing systems, data is constantly transferred between the computation and memory units. This so-called memory bottleneck limits their energy efficiency and speed. In contrast, computation and memory in the human brain (neurons and synapses) are closely and densely interconnected. This gives rise to the brain’s extremely low power consumption at ~20W. Inspired by the brain, neuromorphic computing and artificial neural networks have recently attracted immense interest. In particular, deep neural networks (DNNs) can execute complex processing tasks such as pattern recognition and image reconstruction. However, DNNs are computationally intensive and power hungry. This makes it impractical for them to be scaled up to the level of the complexity for true artificial intelligence (AI). In this project, the team will develop a novel artificial synapse for deep neural networks. This prototypical synapse will offer low power consumption, high precision, good scalability, and great potential for large-scale integration. This work can lead to significant improvement in energy efficiency, bandwidth, and performance for deep learning algorithms. The research outcome can lead to the wide use of AI for both high-performance computing and low-power flexible electronics. This project can revolutionize society through advances in healthcare, self-driving vehicles, and autonomous manufacturing. The team will work closely with their local communities to attract students to pursue engineering careers, especially those from underrepresented groups. Activities will include laboratory demonstrations, design projects, summer internships, and career workshops.Technical:The objective of this project is to develop scalable electrochemical two-dimensional (2D) synaptic arrays with high-precision and low-power for advanced hardware acceleration of deep neural networks (DNNs) with orders of magnitude improvements in energy and speed. While binary SRAM cells have shown promising performance for DNN hardware acceleration, its inherent limitations in power and area make it impractical to scale up to the complexity level required for large-scale problems and/or datasets. In this project, the team will take a holistic approach to develop scalable electrochemical 2D synaptic arrays with high precision, lower-power, good linearity, low variations, and CMOS compatibility for large-scale integration. The team will carry out the following three research tasks: (1) device-level optimization in device precision, dynamic range, and scaling; (2) array-level demonstration by building synaptic arrays, lowering device variations, and designing peripheral circuits; (3) system-level integration via building device models, implementing computing-in-memory (CIM), and demonstrating on-chip learning for pixel-to-pixel applications. This work will provide a low-power and scalable framework for the hardware acceleration of DNNs, paving the ways towards the ubiquitous use of artificial intelligence (AI) in both high-performance computers and low-power embedded systems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

非技术性：大数据革命创造了对新计算范式的迫切需求，以从大型数据集中有效提取有价值的信息。在现有的计算系统中，数据在计算和内存单元之间不断传输。这种所谓的内存瓶颈限制了他们的能源效率和速度。相比之下，人脑中的计算和记忆（神经元和突触）紧密相互联系。这引起了大脑在约20周的极低功耗。受大脑的启发，神经形态计算和人造神经元网络最近引起了极大的兴趣。特别是，深神经网络（DNN）可以执行复杂的处理任务，例如模式识别和图像重建。但是，DNN在计算密集型和饥饿方面是饥饿的。这使得他们不切实际地扩展到真正人工智能（AI）的复杂性水平。在这个项目中，团队将开发一个新型的人工突触，用于深层神经元网络。该原型突触将提供低功耗，高精度，良好的可扩展性和巨大的大规模集成潜力。这项工作可以导致深度学习算法的能源效率，带宽和性能的显着提高。研究结果可能导致广泛用于高性能计算和低功率柔性电子产品。该项目可以通过医疗保健，自动驾驶汽车和自动制造业的进步来彻底改变社会。该团队将与当地社区紧密合作，以吸引学生从事实践工程职业，尤其是来自代表性不足的团体的工程职业。活动将包括实验室演示，设计项目，暑期实习和职业研讨会。技术：该项目的目的是开发可扩展的电化学二维突触阵列（2D）突触阵列，具有高精度和低功率的高级硬件加速，以提高深度神经网络（DNNS）（DNNS），并提高能量和速度的速度。虽然二进制SRAM单元显示了DNN硬件加速度的承诺性能，但其功率和区域的继承限制使得扩展到大规模问题和/或数据集所需的复杂性水平是不切实际的。在这个项目中，团队将采用一种整体方法来开发可扩展的电化学2D合成阵列，具有高精度，较低功率，良好的线性，较低的变化和CMOS兼容性以进行大规模集成。团队将执行以下三项研究任务：（1）设备精度，动态范围和缩放的设备级优化； （2）通过构建合成阵列，降低设备变化和设计外围电路来进行阵列级别的演示； （3）通过构建设备模型进行系统级集成，实现内存计算（CIM），并为像素到像素应用程序演示芯片学习。这项工作将为DNN的硬件加速提供一个低功率和可扩展的框架，粘贴了在高性能计算机和低功率嵌入式系统中无处不在使用人工智能（AI）的方法。这项奖项反映了NSF的法定任务，并通过评估了基金会的智力效果，并通过评估了基金会的范围。

项目成果

On Building Efficient and Robust Neural Network Designs

• DOI: 10.1109/ieeeconf56349.2022.10051891
• 发表时间: 2022-10
• 期刊: 2022 56th Asilomar Conference on Signals, Systems, and Computers
• 作者: Xiaoxuan Yang;Huanrui Yang;Jingchi Zhang;Hai Helen Li;Yiran Chen

HERO: hessian-enhanced robust optimization for unifying and improving generalization and quantization performance

HERO：hessian 增强的鲁棒优化，用于统一和提高泛化和量化性能

• DOI: 10.1145/3489517.3530678
• 发表时间: 2022
• 期刊: The 59th ACM/IEEE Design Automation Conference
• 作者: Yang, Huanrui;Yang, Xiaoxuan;Gong, Neil Zhenqiang;Chen, Yiran
• 通讯作者: Chen, Yiran

SpikeSen: Low-Latency In-Sensor-Intelligence Design With Neuromorphic Spiking Neurons

SpikeSen：具有神经形态尖峰神经元的低延迟传感器内智能设计

• DOI: 10.1109/tcsii.2023.3235888
• 发表时间: 2023
• 期刊: IEEE Transactions on Circuits and Systems II: Express Briefs
• 作者: Li, Ziru;Zheng, Qilin;Chen, Yiran;Li, Hai
• 通讯作者: Li, Hai

Photonic Bayesian Neural Network Using Programmed Optical Noises

• DOI: 10.1109/jstqe.2022.3217819
• 发表时间: 2023-03
• 期刊: IEEE Journal of Selected Topics in Quantum Electronics
• 影响因子: 4.9
• 作者: Changming Wu;Xiaoxuan Yang;Yiran Chen;Mo Li

DyNNamic: Dynamically Reshaping, High Data-Reuse Accelerator for Compact DNNs

• DOI: 10.1109/tc.2022.3184272
• 发表时间: 2023-03
• 期刊: IEEE Transactions on Computers
• 影响因子: 3.7
• 作者: Edward Hanson;Shiyu Li;Xuehai Qian;H. Li;Yiran Chen