AitF: Collaborative Research: A Framework of Simultaneous Acceleration and Storage Reduction on Deep Neural Networks Using Structured Matrices

AitF：协作研究：使用结构化矩阵的深度神经网络同时加速和存储减少的框架

基本信息

批准号：
1733834
负责人：
Bo Yuan
金额：
$ 44.81万
依托单位：
CUNY City College
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2017
资助国家：
美国
起止时间：
2017-09-15 至 2018-11-30
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1733834&HistoricalAwards=false
关键词：
AitF Collaborative Research Framework Simultaneous

项目摘要

Deep neural networks (DNNs) have emerged as a class of powerful techniques for learning solutions in a number of challenging problem domains, including computer vision, natural language processing and bioinformatics. These solutions have been enabled mainly because we now have computational accelerators able to sift through the myriad of data required to train a neural network. As the size of DNN models continues to grow, computational and memory resource requirements for training will also grow, limiting deployment of deep learning in many practical applications. Leveraging the theory of structured matrices, this project will develop a general framework for efficient DNN training and inference, providing a significant reduction in algorithmic complexity measures in terms of both computation and storage. The project, if successful, should fundamentally impact a broad class of deep learning applications. It will explore accelerating this new structure for deep learning algorithms targeting emerging accelerator architectures, and will evaluate the benefits of these advances across a number of application domains, including big data analytics, cognitive systems, unmanned vehicles and aerial systems, and wearable devices. The interdisciplinary nature of this project bridges the areas of matrix theory, machine learning, and computer architecture, and will affect education at both Northeastern and CCNY, including the involvement of underrepresented and undergraduate students in the rich array of research tasks. The project will: (1) for the first time, develop a general theoretical framework for structured matrix-based DNN models and perform detailed analysis and investigation of error bounds, convergence, fast training algorithms, etc.; (2) develop low-space-cost and high-speed inference and training schemes for the fully connected layers of DNNs; (3) impose a weight tensor with structure and enable low computational and space cost convolutional layers; (4) develop high-performance and energy-efficient implementations of deep learning systems on high-performance parallel platforms, low-power embedded platforms, as well as emerging computing paradigms and devices; (5) perform a comprehensive evaluation of the proposed approaches on different performance metrics in a variety of platforms. The project will deliver tuned implementations targeting a range of computational platforms, including ASICs, FPGAs, GPUs and cloud servers. The hardware optimizations will focus on producing high-speed and low-cost implementations of deep learning systems.

深度神经网络 (DNN) 已成为一类强大的技术，用于在许多具有挑战性的问题领域中学习解决方案，包括计算机视觉、自然语言处理和生物信息学。这些解决方案之所以能够实现，主要是因为我们现在拥有能够筛选的计算加速器。随着训练神经网络所需的大量数据不断增长，训练的计算和内存资源需求也会增加，从而限制了深度学习在许多实际应用中的部署。这该项目将开发一个 DNN 训练和推理的通用框架，在计算和存储方面显着降低算法复杂性。该项目如果成功，将从根本上影响广泛的深度高效学习应用。这种针对新兴加速器架构的深度学习算法的新结构，并将评估这些进步在许多应用领域的好处，包括大数据分析、认知系统、无人驾驶车辆和航空系统以及可穿戴设备的跨学科性质。项目架起了各地区的桥梁该项目将：（1）首次。，开发基于结构化矩阵的DNN模型的通用理论框架，并对误差范围、收敛性、快速训练算法等进行详细的分析和研究（2）开发低空间成本和高速的推理和训练方案；全连接层DNN；(3) 施加具有结构的权重张量，并启用低计算和空间成本的卷积层；(4) 在高性能并行平台、低功耗嵌入式平台上开发深度学习系统的高性能和节能实现；以及新兴的计算范例和设备；(5) 对各种平台中的不同性能指标进行综合评估该项目将针对一系列计算平台（包括 ASIC、FPGA、 GPU 和云服务器的硬件优化将侧重于深度学习系统的高速和低成本实现。

项目成果

期刊论文数量（5）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

C ir CNN: accelerating and compressing deep neural networks using block-circulant weight matrices

C ir CNN：使用块循环权重矩阵加速和压缩深度神经网络

DOI：
10.1145/3123939.3124552
发表时间：
2017-01
期刊：
Proceedings of the 50th Annual IEEE/ACM International Symposium on Microarchitecture
影响因子：
0
作者：
Ding, Caiwen;Yuan, Geng;Ma, Xiaolong;Zhang, Yipeng;Tang, Jian;Qiu, Qinru;Lin, Xue;Yuan, Bo;Liao, Siyu;Wang, Yanzhi;et al
通讯作者：
et al

C-LSTM: Enabling Efficient LSTM using Structured Compression Techniques on FPGAs

C-LSTM：在 FPGA 上使用结构化压缩技术实现高效 LSTM

DOI：
10.1145/3174243.3174253
发表时间：
2018-02-15
期刊：
Proceedings of the 2018 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays
影响因子：
0
作者：
Shuo Wang;Zhe Li;Caiwen Ding;Bo Yuan;Qinru Qiu;Yanzhi Wang;Yun Liang
通讯作者：
Yun Liang

Implementation of a Near-Optimal Complex Root Clustering Algorithm

近最优复根聚类算法的实现

DOI：
10.1007/978-3-319-96418-8_28
发表时间：
2018-01
期刊：
International Congress on Mathematical Software (ICMS
影响因子：
0
作者：
Imbach, Rémi;Pan, Victor;Yap, Chee
通讯作者：
Yap, Chee

Towards Ultra-High Performance and Energy Efficiency of Deep Learning Systems: An Algorithm-Hardware Co-Optimization Framework

迈向深度学习系统的超高性能和高能效：算法-硬件协同优化框架

DOI：
10.1609/aaai.v32i1.11653
发表时间：
2018-02-18
期刊：
ArXiv
影响因子：
0
作者：
Yanzhi Wang;Caiwen Ding;Zhe Li;Geng Yuan;Siyu Liao;Xiaolong Ma;Bo Yuan;Xuehai Qian;Jian Tang;Qinru Qiu;X. Lin
通讯作者：
X. Lin

Energy-efficient, high-performance, highly-compressed deep neural network design using block-circulant matrices

使用块循环矩阵的节能、高性能、高压缩深度神经网络设计