CSR: Small: Towards Efficient Deep Inference for Mobile Applications

CSR：小：迈向移动应用程序的高效深度推理

• 批准号: 1815619
• 负责人: Tian Guo
• 金额: $ 49.97万
• 依托单位: Worcester Polytechnic Institute
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2018
• 资助国家: 美国
• 起止时间: 2018-07-01 至 2022-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1815619&HistoricalAwards=false
• 关键词: CSR; Small; Towards; Efficient; Deep

An ever-increasing number of mobile applications are using deep learning models to provide novel and useful features, such as language translation and object recognition. These features are supported by passing input data, for example a photo or an audio clip, to complex models in order to generate meaningful output. However, mobile applications that use deep learning models currently need to choose between prediction accuracy and speed at development time. This can lead to poor user experience due to reasons such as running state-of-the-art models on older mobile devices. The proposed MODI (MObile Deep Inference) project outlines new research in designing and implementing a mobile-aware deep inference platform that combines innovations in both algorithm and system optimizations. The proposed work will address mobile deep inference performance problems by enabling flexible, fine-grained model partition and layer-based inference execution, as well as mobile-specific model designs. In addition, MODI enables a scalable mobile deep inference paradigm with efficient model management both on-device and in the cloud. The project will empower deep learning to provide useful features for mobile applications with significantly improved performance. Consequently, this project will open doors to allow running optimized deep learning models on much more resource-constrained devices such as embedded devices. The MODI project can be used as a standalone cloud system or integrated with existing general inference serving platforms by incorporating its mobile-specific optimizations, thereby increasing adoption. The broader impacts of the project will include graduate and undergraduate courses that incorporate research results, outreach to expose undergraduates and K-12 students to research in both computer systems and deep learning. In addition, project related source code and other resources will be released to the research community through the project website at http://tianguo.info/projects/modi.htmlThis award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

越来越多的移动应用程序正在使用深度学习模型来提供新颖且有用的功能，例如语言翻译和对象识别。这些功能是通过将输入数据（例如照片或音频剪辑）传递到复杂模型以生成有意义的输出来支持的。然而，使用深度学习模型的移动应用程序目前需要在开发时在预测精度和速度之间进行选择。由于在较旧的移动设备上运行最先进的模型等原因，这可能会导致用户体验不佳。拟议的 MODI（移动深度推理）项目概述了设计和实现移动感知深度推理平台的新研究，该平台结合了算法和系统优化方面的创新。所提出的工作将通过实现灵活、细粒度的模型划分和基于层的推理执行以及特定于移动设备的模型设计来解决移动深度推理性能问题。此外，MODI 支持可扩展的移动深度推理范例，并在设备上和云端进行高效的模型管理。该项目将使深度学习能够为移动应用程序提供有用的功能，并显着提高性能。因此，该项目将为允许在资源更加受限的设备（例如嵌入式设备）上运行优化的深度学习模型打开大门。 MODI 项目可以用作独立的云系统，也可以通过整合其针对移动设备的优化来与现有的通用推理服务平台集成，从而提高采用率。该项目的更广泛影响将包括纳入研究成果的研究生和本科生课程，以及让本科生和 K-12 学生接触计算机系统和深度学习研究的外展活动。此外，项目相关源代码和其他资源将通过项目网站http://tianguo.info/projects/modi.html向研究界发布。该奖项体现了NSF的法定使命，经评估认为值得支持。基金会的智力价值和更广泛的影响审查标准。

项目成果

PERSEUS: Characterizing Performance and Cost of Multi-Tenant Serving for CNN Models

• DOI: 10.1109/ic2e48712.2020.00014
• 发表时间: 2020-01-01
• 期刊: 2020 IEEE INTERNATIONAL CONFERENCE ON CLOUD ENGINEERING (IC2E 2020)
• 作者: LeMay, Matthew;Li, Shijian;Guo, Tian
• 通讯作者: Guo, Tian

MDINFERENCE: Balancing Inference Accuracy and Latency for Mobile Applications

• DOI: 10.1109/ic2e48712.2020.00010
• 发表时间: 2020-02
• 期刊: 2020 IEEE International Conference on Cloud Engineering (IC2E)
• 作者: Samuel S. Ogden;Tian Guo

EPNet: Learning to Exit with Flexible Multi-Branch Network

EPNet：学习通过灵活的多分支网络退出

• DOI: 10.1145/3340531.3411973
• 发表时间: 2020
• 期刊: ACM International Conference on Information and Knowledge Management (CIKM'20
• 作者: Dai, Xin;Kong, Xiangnan;Guo, Tian
• 通讯作者: Guo, Tian

An Experimental Evaluation of Garbage Collectors on Big Data Applications

垃圾收集器大数据应用的实验评估

• DOI: 10.14778/3303753.3303762
• 发表时间: 2019
• 期刊: Proceedings of the VLDB Endowment (VLDB 2019)
• 作者: Lijie Xu;Tian Guo;Wensheng Dou;Wei Wang;Jun Wei
• 通讯作者: Jun Wei

Sync-Switch: Hybrid Parameter Synchronization for Distributed Deep Learning

• DOI: 10.1109/icdcs51616.2021.00057
• 发表时间: 2021-04
• 期刊: 2021 IEEE 41st International Conference on Distributed Computing Systems (ICDCS)
• 作者: Shijian Li;Oren Mangoubi;Lijie Xu;Tian Guo