CRII: SHF: Efficiency-Aware Robust Implementation of Neural Networks with Algorithm-Hardware Co-design

CRII：SHF：具有算法硬件协同设计的神经网络的效率感知稳健实现

基本信息

批准号：
1947826
负责人：
Priyadarshini Panda
金额：
$ 17.5万
依托单位：
Yale University
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2020
资助国家：
美国
起止时间：
2020-04-01 至 2023-03-31
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1947826&HistoricalAwards=false
关键词：
CRII SHF Efficiency Aware Robust

项目摘要

With the advent of Internet-of-Things and the necessity to enable intelligence in embedded devices like mobile phones, wearables etc., low-power and secure hardware implementation of neural networks is vital. Despite achieving high performance and unprecedented classification accuracies on a variety of perception tasks, Deep Neural Networks (DNNs) have been shown to be adversarially vulnerable. For example, a DNN can be easily fooled into mis-classifying an input with slight changes of image-pixel intensities. This vulnerability severely limits the deployment and its use in safety-critical real-world tasks such as self-driving cars, malware detection, healthcare monitoring systems etc. This project investigates hardware aware techniques to resolve or resist software vulnerabilities (specifically, adversarial attacks) by exploring the design space of energy-accuracy-robustness trade-off cohesively with algorithm-hardware co-design to create functional intelligent systems. Thus, the project seeks to develop robustness-aware algorithms broadly applicable to the energy-efficient and secure implementation of DNN engines on both current CMOS accelerator platforms and emerging memory technologies. Furthermore, the research will support the interdisciplinary development of a diverse cohort of PhD and undergraduate students, and the development of a graduate-level course at Yale University on neural network architectures and learning algorithms tied with robustness from circuit and system design perspective.The technical aims of this project are divided into two thrusts. The first thrust develops robustness centred algorithms in DNNs where techniques such as quantization, pruning among others are used to improve the adversarial resilience of models while yielding energy-efficiency benefits. This part also identifies a new form of noise stability for DNNs, i.e., the sensitivity of each layer’s computation to adversarial noise. This allows for a principled way of applying layer-specific algorithmic modifications that incurs adversarial robustness as well as energy-efficiency with minimal loss in accuracy. The second thrust benchmarks and implements the proposed robust computing models on emerging technology-based memristor crossbar-array platforms to investigate the hardware-level benefits (while comparing with standard CMOS accelerator baselines). In particular, design issues and complexities for implementing variable precision, stochastic and combined stochastic-deterministic neuronal activity will be investigated. The two thrusts offer a fundamental co-design infrastructure where algorithmic innovations will be used to optimize robust and efficient hardware implementations for neural networks.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

随着物联网的出现以及在移动电话、可穿戴设备等嵌入式设备中实现智能的必要性，尽管在各种方面实现了高性能和前所未有的分类精度，但神经网络的低功耗和安全硬件实现仍然至关重要。在感知任务中，深度神经网络 (DNN) 已被证明是容易受到攻击的，例如，DNN 很容易因图像像素强度的微小变化而对输入进行错误分类，这种严重的漏洞限制了这种能力。部署及其在安全关键的现实任务中的使用，例如自动驾驶汽车、恶意软件检测、医疗保健监控系统等。该项目通过探索设计空间来研究硬件感知技术，以解决或抵御软件漏洞（特别是对抗性攻击）因此，该项目旨在开发广泛适用于 DNN 引擎的节能和安全实施的鲁棒性感知算法。此外，该研究还将支持不同群体的博士生和本科生的跨学科发展，以及耶鲁大学关于神经网络架构和学习算法的研究生课程的开发。从电路和系统设计的角度来看具有鲁棒性。该项目的技术目标分为两个主旨，第一个主旨是开发 DNN 中以鲁棒性为中心的算法，其中使用量化、剪枝等技术来提高模型的对抗能力。这部分还确定了 DNN 的一种新形式的噪声稳定性，即每层计算对对抗性噪声的敏感性，这允许采用一种原则性的方法来应用特定于层的算法修改，从而产生对抗性鲁棒性。第二个推力基准测试并在基于新兴技术的忆阻器交叉阵列平台上实现所提出的鲁棒计算模型，研究硬件级别。特别是，将研究实现可变精度、随机和组合随机确定性神经活动的设计问题和复杂性，这两个方向提供了算法创新的基本协同设计基础设施。用于优化神经网络的稳健和高效的硬件实现。该奖项反映了 NSF 的法定使命，并且通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。