CRII: CNS: A Systematic Multi-Task Learning Framework for Improving Deep Learning Efficiency on Edge Platforms

CRII：CNS：用于提高边缘平台深度学习效率的系统多任务学习框架

• 批准号: 2245765
• 负责人: Tianyun Zhang
• 金额: $ 17.42万
• 依托单位: Cleveland State University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-06-01 至 2025-05-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2245765&HistoricalAwards=false
• 关键词: CRII; CNS; Systematic; Multi; Task

Multi-task learning is a subfield of machine learning in which the data is trained with a shared model to solve different tasks simultaneously. Multi-task learning highly reduces the number of parameters in the machine learning models and thus reduces the computational and storage requirements. For example, there are multiple tasks to be done in real-time in self-driving cars, including object detection and depth estimation. If these tasks can be trained on a single model with shared parameters, the model size and the inference time can be highly reduced. This project aims to further compress the model used for multi-task learning as the model size of a single deep neural network is still a critical challenge to many computation systems, especially for edge platforms. This project proposes an approach to learn the difficulty of every task and maintain the performance of the most difficult task when compressing a multi-task learning model. It increases the potential in the compression rate with acceptable performance for all the tasks as the performance of the most difficult task needs to be guaranteed to provide a satisfactory user experience. This project also designs an efficient multi-task federated learning approach for edge platforms. It improves the convergence rate of multi-task federated learning and reduces the communication costs in every iteration. Finally, this project proposes to solve an algorithm-hardware co-design problem to maximize the implementation efficiency of the compressed multi-task DNN models on edge platforms.The files of compressed DNN models and the ideas on efficient DNN training and implementation may be useful to researchers who focus on improving the computation efficiency of DNN models on edge platforms and other hardware platforms.This project will involve undergraduate and graduate students in the research. The research achievements of this project will be incorporated into a current senior-level undergraduate course, a new planned advanced-level graduate course, and seminars for both undergraduate and graduate students. There are also planned research demonstrations during the workshops and summer camps for the K-12 students.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

多任务学习是机器学习的一个子字段，其中数据经过共享模型培训，以同时解决不同的任务。多任务学习高度减少了机器学习模型中的参数数量，从而减少了计算和存储要求。例如，在自动驾驶汽车中实时执行多个任务，包括对象检测和深度估计。如果可以在具有共享参数的单个模型上训练这些任务，则可以大大减少模型大小和推理时间。该项目旨在进一步压缩用于多任务学习的模型，因为单个深神经网络的模型大小仍然是许多计算系统的关键挑战，尤其是对于边缘平台。该项目提出了一种学习每项任务难度的方法，并在压缩多任务学习模型时保持最困难的任务的性能。它增加了压缩率的潜力，所有任务的可接受性能，因为需要保证最困难的任务的性能以提供令人满意的用户体验。该项目还为边缘平台设计了有效的多任务联合学习方法。它提高了多任务联合学习的融合率，并降低了每次迭代的沟通成本。最后，该项目提议解决算法 - 硬件共同设计问题，以最大化边缘平台上压缩的多任务DNN模型的实现效率。压缩DNN模型的文件以及有效的DNN培训和实施的想法可能是有用的。对于专注于提高边缘平台和其他硬件平台上DNN模型的计算效率的研究人员而言。该项目将涉及本科生和研究生。该项目的研究成就将纳入当前的高级本科课程，新计划的高级研究生课程以及本科生和研究生的研讨会。 在针对K-12学生的研讨会和夏令营期间，也有计划的研究演示。该奖项反映了NSF的法定任务，并使用基金会的知识分子优点和更广泛的影响审查标准，被认为值得通过评估。