CNS Core: Small: Harnessing Weight Repetition for Efficient Deep Neural Network Inference on General-Purpose Platforms

CNS 核心：小型：利用权重重复在通用平台上进行高效深度神经网络推理

• 批准号: 1909999
• 负责人: Christopher Fletcher
• 金额: $ 50万
• 依托单位: University of Illinois at Urbana-Champaign
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-10-01 至 2023-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1909999&HistoricalAwards=false
• 关键词: CNS; Core; Small; Harnessing; Weight

Society is witnessing an explosion in the use of Deep Neural Networks (DNNs) across all facets of daily life including health, finances, entertainment and transportation. DNNs are used by performing DNN inference, which queries the DNN with an input (for example, an image) to get an answer (for example, a classification). Society relies on inference every day, where it is run on devices ranging from cloud servers to personal computers. The goal of this project is to develop new ways to make inference efficient (fast, low power) on these devices.The technical approach is to explore how a new phenomenon, called weight repetition, can be applied to general-purpose devices such as Central Processing Units (CPUs) and Graphical Processing Units (GPUs). The idea is, when a DNN weight is repeated, DNN inference operations can be simplified. The first project thrust will develop high-efficiency weight repetition-aware software kernels that can run on un-modified hardware. The second thrust will develop novel training techniques to co-design the DNN with the weight repetition-aware kernels. Finally, the third thrust will explore what point hardware modifications can be made to further improve efficiency in the first two thrusts.By proving weight repetition's effectiveness on general-purpose devices, this project will unlock innovation in software, algorithms and hardware. The project will also amplify the improvement possible from related, but orthogonal, techniques such as weight quantization and weight sparsity. To support the cross-stack approach, the project will train a new class of students and researchers who can work across high-performance software, hardware and DNN training algorithms to build co-designed Machine Learning stacks and, in the future, apply the lessons learned to other high-impact problems that require cross-layer solutions.The project will store all publications, code and data-sets on public-facing websites, hosted at the University of Illinois for at least 3 years after the end of the project. This information will be made available via commercial websites. Links to these websites will be mirrored at http://cwfletcher.net/weightrepetition.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

社会正在见证深度神经网络 (DNN) 在日常生活各个方面的使用呈爆炸式增长，包括健康、金融、娱乐和交通。 DNN 通过执行 DNN 推理来使用，即使用输入（例如图像）查询 DNN 以获得答案（例如分类）。 社会每天都依赖于推理，推理运行在从云服务器到个人电脑的各种设备上。 该项目的目标是开发新方法，使这些设备上的推理高效（快速、低功耗）。技术方法是探索如何将一种称为权重重复的新现象应用于 Central 等通用设​​备处理单元 (CPU) 和图形处理单元 (GPU)。 这个想法是，当重复 DNN 权重时，可以简化 DNN 推理操作。 第一个项目主旨将开发可在未修改的硬件上运行的高效权重重复感知软件内核。 第二个重点将开发新颖的训练技术，以共同设计 DNN 和权重重复感知内核。 最后，第三个主旨将探讨在前两个主旨中可以对哪些硬件进行修改，以进一步提高效率。通过证明权重重复在通用设备上的有效性，该项目将释放软件、算法和硬件方面的创新。 该项目还将扩大相关但正交的技术（例如权重量化和权重稀疏性）可能带来的改进。 为了支持跨堆栈方法，该项目将培训一批新的学生和研究人员，他们可以跨高性能软件、硬件和 DNN 训练算法进行工作，以构建共同设计的机器学习堆栈，并在未来应用所学到的经验教训了解其他需要跨层解决方案的高影响力问题。该项目将在项目结束后将所有出版物、代码和数据集存储在伊利诺伊大学托管的面向公众的网站上至少 3 年。 该信息将通过商业网站提供。 这些网站的链接将反映在 http://cwfletcher.net/weightrepetition。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Speculative Taint Tracking (STT): A Comprehensive Protection for Speculatively Accessed Data

推测性污点跟踪 (STT)：对推测性访问数据的全面保护

• DOI: 10.1145/3352460.3358274
• 发表时间: 2019-10
• 期刊: International Conference on Microarchitecture
• 作者: Yu, Jiyong;Yan, Mengjia;Khyzha, Artem;Morrison, Adam;Torrellas, Josep;Fletcher, Christopher W.
• 通讯作者: Fletcher, Christopher W.