Algorithms and Theory for Compressing Deep Neural Networks

压缩深度神经网络的算法和理论

基本信息

批准号：
2208126
负责人：
Penghang Yin
金额：
$ 27.66万
依托单位：
SUNY at Albany
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2022
资助国家：
美国
起止时间：
2022-09-01 至 2025-08-31
项目状态：
未结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2208126&HistoricalAwards=false
关键词：
Algorithms Theory Compressing Deep Neural

项目摘要

Deep neural networks (DNNs) have been the main driving force for recent advancements in artificial intelligence (AI) technology, profoundly impacting society in the areas of transportation, public safety, entertainment, health care, and other areas of public life. One of the biggest obstacles to AI's even broader impact on our daily lives is the typically enormous power consumption of DNNs upon deployment. The aim of this project is to develop mathematical and computational approaches for DNN compression to realize the fast and efficient deployment of AI systems on mobile platforms with low-power budgets such as smartphones. Results of this work will have a variety of applications which include video security systems, autopilot, smart robots, and face identification. The project will involve training of graduate students, development of data science courses, as well as collaboration with industry. The PI plans to (1) develop and analyze coarse gradient algorithms, featuring a biased first-order oracle, for the discretization of various neural architectures including transformer-based networks; (2) develop and analyze efficient thresholding-based algorithms for compressing networks via structured sparsity on both balanced and unbalanced data; (3) investigate the model capacity of compressed DNNs and establish universal finite-sample expressivity theory. The proposed research will also explore the applications of coarse gradient algorithms to other machine learning problems with discrete-valued loss functions and advance knowledge in discrete optimization.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

深度神经网络（DNNS）一直是人工智能（AI）技术进步的主要驱动力，在运输，公共安全，娱乐，卫生保健和其他公共生活领域深远影响社会。 AI对我们日常生活的更广泛影响的最大障碍之一是部署时DNN的巨大功耗。该项目的目的是开发DNN压缩的数学和计算方法，以实现在具有智能手机等低功率预算的移动平台上快速有效地部署AI系统。这项工作的结果将包含各种应用程序，包括视频安全系统，自动驾驶仪，智能机器人和面部识别。该项目将涉及培训研究生，数据科学课程的发展以及与行业的合作。 PI计划（1）开发和分析具有偏见的一阶甲骨文的粗梯度算法，以离散各种神经体系结构，包括基于变压器的网络；（2）开发和分析有效的基于阈值的算法，以通过在平衡和不平衡数据上通过结构化稀疏来压缩网络；（3）研究压缩DNN的模型能力并建立了通用的有限样本表达理论。拟议的研究还将探讨粗梯度算法在其他机器学习问题上的应用，并具有离散价值的损失功能并提高离散优化方面的知识。该奖项反映了NSF的法定任务，并被认为值得通过基金会的知识分子优点和更广泛的审查标准来通过评估来获得支持。