Collaborative Research: CNS Core: Small: Towards Automated and QoE-driven Machine Learning Model Selection for Edge Inference

合作研究：CNS 核心：小型：面向边缘推理的自动化和 QoE 驱动的机器学习模型选择

• 批准号: 2006630
• 负责人: Jie Xu
• 金额: $ 25万
• 依托单位: University of Miami
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-10-01 至 2024-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2006630&HistoricalAwards=false
• 关键词: Collaborative; Research; CNS; Core; Small

Edge devices, such as mobile phones, drones and robots, have been emerging as an increasingly more important platform for deep neural network (DNN) inference. For an edge device, selecting an optimal DNN model out of many possibilities is crucial for maximizing the user’s quality of experience (QoE), but this is significantly challenged by the high degree of heterogeneity in edge devices and constant-changing usage scenarios. The current practice commonly selects a single DNN model for many or all edge devices, which can only provide a satisfactory QoE for a small fraction of users at best. Alternatively, device-specific DNN model optimization is time-consuming and not scalable to a large diversity of edge devices. Moreover, the existing approaches focus on optimizing a certain objective metric for edge inference, which may not translate into improvement of the actual QoE for users. By leveraging the predictive power of machine learning and keeping users in a loop, this project proposes an automated and scalable device-level DNN model selection engine for QoE-optimal edge inference. Specifically, this project includes two thrusts: first, it exploits online learning to predict QoE for each edge device, automating deployment-stage DNN model selection; and second, it builds a runtime QoE predictor and automatically selects an optimal DNN model given runtime contextual information.This project represents an important departure from and an essential complement to the current practices in DNN model optimization. It can bring the benefits of DNN-enabled intelligence to many more resource-constrained edge devices with an optimal QoE. Additionally, it provides novel observations, insights and principles for edge inference, catalyzing the transformation of the design of DNN models into a new user-centric paradigm. This project also enables new opportunities to improve curriculum design and attract students, especially under-represented minorities, to engage in science, technology, engineering, and mathematics fields.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

手机、无人机和机器人等边缘设备已成为越来越重要的深度神经网络 (DNN) 推理平台。对于边缘设备来说，从多种可能性中选择最佳的 DNN 模型对于最大化用户的能力至关重要。体验质量（QoE），但由于边缘设备的高度异构性和不断变化的使用场景，当前的实践通常为许多或所有边缘设备选择单一的 DNN 模型，这只能提供令人满意的效果。体验质量或者，对于一小部分用户来说，特定于设备的 DNN 模型优化非常耗时，并且无法扩展到大量边缘设备。此外，现有方法侧重于优化边缘推理的某些客观指标。通过利用机器学习的预测能力并使用户处于循环状态，该项目提出了一种自动化且可扩展的设备级 DNN 模型选择引擎，用于 QoE 最佳边缘推理。该项目包括两个重点：首先，它利用在线学习来预测每个边缘设备的 QoE，自动选择部署阶段的 DNN 模型；其次，它构建一个运行时 QoE 预测器，并在给定运行时上下文信息的情况下自动选择最佳 DNN 模型。它是对当前 DNN 模型优化实践的重要偏离和重要补充，它可以为更多资源受限的边缘设备带来支持 DNN 的智能的优势，并提供最佳的 QoE。该项目还提供了改进课程设计并吸引学生（尤其是代表性不足的少数群体）参与科学的新机会。 、技术、工程和数学领域。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Improving QoE of Deep Neural Network Inference on Edge Devices: A Bandit Approach

• DOI: 10.1109/jiot.2022.3182728
• 发表时间: 2022-11
• 期刊: IEEE Internet of Things Journal
• 影响因子: 10.6
• 作者: Bingqian Lu;Jianyi Yang;Jie Xu;Shaolei Ren

Adaptive Deep Neural Network Ensemble for Inference-as-a-Service on Edge Computing Platforms

用于边缘计算平台上的推理即服务的自适应深度神经网络集成

• DOI: 10.1109/mass52906.2021.00013
• 发表时间: 2021
• 期刊: 2021 IEEE 18th International Conference on Mobile Ad Hoc and Smart Systems (MASS
• 作者: Bai, Yang;Chen, Lixing;Zhang, Letian;Abdel-Mottaleb, Mohamed;Xu, Jie
• 通讯作者: Xu, Jie

Automated Ensemble for Deep Learning Inference on Edge Computing Platforms

• DOI: 10.1109/jiot.2021.3102945
• 发表时间: 2022-03
• 期刊: IEEE Internet of Things Journal
• 影响因子: 10.6
• 作者: Yang Bai;Lixing Chen;M. Abdel-Mottaleb;Jie Xu

Bandwidth Allocation for Multiple Federated Learning Services in Wireless Edge Networks

• DOI: 10.1109/twc.2021.3113346
• 发表时间: 2021-01
• 期刊: IEEE Transactions on Wireless Communications
• 影响因子: 10.4
• 作者: Jie Xu;Heqiang Wang;Lixing Chen

NeuE: Automated Neural Network Ensembles for Edge Intelligence

• DOI: 10.1109/tetc.2022.3214931
• 发表时间: 2023-04
• 期刊: IEEE Transactions on Emerging Topics in Computing
• 影响因子: 5.9
• 作者: Yang Bai;Lixing Chen;Jie Xu