CSR: Small: Towards Efficient Deep Inference for Mobile Applications

CSR：小：迈向移动应用程序的高效深度推理

• 批准号: 1815619
• 负责人: Tian Guo
• 金额: $ 49.97万
• 依托单位: Worcester Polytechnic Institute
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-07-01 至 2022-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1815619&HistoricalAwards=false
• 关键词: CSR; Small; Towards; Efficient; Deep

An ever-increasing number of mobile applications are using deep learning models to provide novel and useful features, such as language translation and object recognition. These features are supported by passing input data, for example a photo or an audio clip, to complex models in order to generate meaningful output. However, mobile applications that use deep learning models currently need to choose between prediction accuracy and speed at development time. This can lead to poor user experience due to reasons such as running state-of-the-art models on older mobile devices. The proposed MODI (MObile Deep Inference) project outlines new research in designing and implementing a mobile-aware deep inference platform that combines innovations in both algorithm and system optimizations. The proposed work will address mobile deep inference performance problems by enabling flexible, fine-grained model partition and layer-based inference execution, as well as mobile-specific model designs. In addition, MODI enables a scalable mobile deep inference paradigm with efficient model management both on-device and in the cloud. The project will empower deep learning to provide useful features for mobile applications with significantly improved performance. Consequently, this project will open doors to allow running optimized deep learning models on much more resource-constrained devices such as embedded devices. The MODI project can be used as a standalone cloud system or integrated with existing general inference serving platforms by incorporating its mobile-specific optimizations, thereby increasing adoption. The broader impacts of the project will include graduate and undergraduate courses that incorporate research results, outreach to expose undergraduates and K-12 students to research in both computer systems and deep learning. In addition, project related source code and other resources will be released to the research community through the project website at http://tianguo.info/projects/modi.htmlThis award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

越来越多的移动应用程序正在使用深度学习模型来提供新颖而有用的功能，例如语言翻译和对象识别。这些功能通过将输入数据（例如照片或音频剪辑）传递给复杂模型来支持这些功能，以生成有意义的输出。但是，当前使用深度学习模型的移动应用程序需要在开发时预测准确性和速度之间进行选择。由于原因，例如在较旧的移动设备上运行最新模型，这可能会导致用户体验差。拟议的MODI（移动深度推理）项目概述了设计和实施移动感知的深度推理平台的新研究，该平台结合了算法和系统优化方面的创新。拟议的工作将通过启用灵活的，细粒度的模型分区和基于层的推理执行以及特定于移动的模型设计来解决移动推理性能问题。此外，MODI启用具有有效模型管理的可扩展移动深度推理范式，无论是在设备上还是在云中。该项目将授权深度学习为移动应用程序提供有用的功能，并具有大大改善的性能。因此，该项目将打开门，以允许在更具资源约束设备（例如嵌入式设备）上运行优化的深度学习模型。 MODI项目可以用作独立的云系统，也可以通过合并其移动特定优化来与现有的一般推理服务平台集成，从而增加采用。该项目的更广泛影响将包括毕业和本科课程，这些课程结合了研究结果，宣传以揭露本科生和K-12学生，以研究计算机系统和深度学习。此外，与项目相关的源代码和其他资源将通过项目网站http://tianguo.info/projects/modi.htmlthis奖将其发布给研究社区，反映了NSF的法定任务，并被认为是值得通过基金会的智力和更广泛影响的评估来通过评估来获得支持的。

项目成果

PERSEUS: Characterizing Performance and Cost of Multi-Tenant Serving for CNN Models

• DOI: 10.1109/ic2e48712.2020.00014
• 发表时间: 2020-01-01
• 期刊: 2020 IEEE INTERNATIONAL CONFERENCE ON CLOUD ENGINEERING (IC2E 2020)
• 作者: LeMay, Matthew;Li, Shijian;Guo, Tian
• 通讯作者: Guo, Tian

MDINFERENCE: Balancing Inference Accuracy and Latency for Mobile Applications

• DOI: 10.1109/ic2e48712.2020.00010
• 发表时间: 2020-02
• 期刊: 2020 IEEE International Conference on Cloud Engineering (IC2E)
• 作者: Samuel S. Ogden;Tian Guo

EPNet: Learning to Exit with Flexible Multi-Branch Network

EPNet：学习通过灵活的多分支网络退出

• DOI: 10.1145/3340531.3411973
• 发表时间: 2020
• 期刊: ACM International Conference on Information and Knowledge Management (CIKM'20
• 作者: Dai, Xin;Kong, Xiangnan;Guo, Tian
• 通讯作者: Guo, Tian

An Experimental Evaluation of Garbage Collectors on Big Data Applications

垃圾收集器大数据应用的实验评估

• DOI: 10.14778/3303753.3303762
• 发表时间: 2019
• 期刊: Proceedings of the VLDB Endowment (VLDB 2019)
• 作者: Lijie Xu;Tian Guo;Wensheng Dou;Wei Wang;Jun Wei
• 通讯作者: Jun Wei

Sync-Switch: Hybrid Parameter Synchronization for Distributed Deep Learning

• DOI: 10.1109/icdcs51616.2021.00057
• 发表时间: 2021-04
• 期刊: 2021 IEEE 41st International Conference on Distributed Computing Systems (ICDCS)
• 作者: Shijian Li;Oren Mangoubi;Lijie Xu;Tian Guo