CCSS: Collaborative Research: Quality-Aware Distributed Computation for Wireless Federated Learning: Channel-Aware User Selection, Mini-Batch Size Adaptation, and Scheduling

CCSS：协作研究：无线联邦学习的质量感知分布式计算：通道感知用户选择、小批量大小自适应和调度

基本信息

批准号：
2203238
负责人：
Junshan Zhang
金额：
$ 22万
依托单位：
University of California-Davis
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2021
资助国家：
美国
起止时间：
2021-10-01 至 2024-07-31
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2203238&HistoricalAwards=false
关键词：
CCSS Collaborative Research Quality Aware

项目摘要

With the explosive growth of ML/AI technologies, there is enormous potential to advance networking technologies to enable distributed ML/AI data analytics over networked systems. This project will explore innovative cross-disciplinary research at the intersections of wireless networking and machine learning, and study wireless federated learning (FL) for achieving collaborative intelligence in wireless networks. It will advance the fundamental understanding of quality-aware dynamic distributed computation and computation-communication co-design for wireless FL. This project will spur a new line of thinking and provide new insights to support various emerging ML/AI applications over wireless networked systems, such as collaborative robotics, multi-user mixed reality, and intelligent control and management of wireless networks. The proposed research will also be integrated with education activities at the PIs' institutions for graduate, undergraduate, and K-12 students via curriculum development, research experiences, and outreach. The PIs will make conscientious effort to recruit minority graduate students.This project will study quality-aware distributed computation for wireless FL, with focuses on channel-aware user selection, communication scheduling, and adaptive mini-batch size design. The proposed research is built on the key observation that the learning accuracy of the trained model in FL depends heavily on dynamic selection of users participating in the learning process and the quality of their local model updates (which is determined by their mini-batch sizes). The quality of local updates can be treated as a design parameter and used as a knob for adaptive control across users and over time based on users' communication and computation costs as well as capabilities. With this insight, the PIs will 1) quantify the impacts of the variances of users' local stochastic gradient updates on learning accuracy over the learning process, for general settings including non-IID data, non-convex loss functions, and asynchronous distributed learning; 2) develop adaptive algorithms that select the participating users and set their mini-batch sizes in each round of the FL algorithm, based on users' channel conditions and the impacts of their local updates on the training loss; 3) jointly design users' mini-batch sizes and schedule their communications to reduce the learning time, by investigating the intricate coupling between computation workloads and communication scheduling. Multi-objective optimization will be used to strike the right balance between learning accuracy and learning cost (or learning time).This project is jointly funded by the Division of Electrical, Communications and Cyber Systems (ECCS), and the Established Program to Stimulate Competitive Research (EPSCoR).This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

随着ML/AI技术的爆炸性增长，具有巨大的潜力来推进网络技术，以在网络系统上启用分布式ML/AI数据分析。该项目将在无线网络和机器学习的交集中探索创新的跨学科研究，并研究无线联合学习（FL），以实现无线网络中的协作智能。它将提高对无线FL的质量感知动态分布式计算和计算通信共设计的基本理解。该项目将刺激新的思维方式，并提供新的见解，以支持无线网络系统上各种新兴的ML/AI应用程序，例如协作机器人技术，多用户混合现实以及对无线网络的智能控制和管理。拟议的研究还将通过课程开发，研究经验和宣传来与PIS的研究生，本科生和K-12学生的教育活动相结合。 PI将尽职尽责地招募少数群体研究生。该项目将研究无线FL的质量意识分布式计算，重点关注渠道吸引的用户选择，通信调度和自适应迷你批量设计。拟议的研究基于关键观察，即FL中受过训练的模型的学习准确性在很大程度上取决于参与学习过程的用户的动态选择以及其本地模型更新的质量（这取决于他们的迷你批量尺寸）。可以将本地更新的质量视为设计参数，并用作跨用户自适应控制的旋钮，并且根据用户的通信和计算成本以及功能。有了这种见解，PIS将1）量化用户本地随机梯度方差对学习过程中学习准确性的影响，以适用于包括非IID数据，非Convex损失函数以及异步分布式学习的一般环境； 2）开发自适应算法，以选择参与的用户，并根据用户的渠道条件以及其本地更新对培训损失的影响； 3）通过调查计算工作负载和通信计划之间的复杂耦合，共同设计用户的迷你批量大小，并安排他们的通信以减少学习时间。多目标优化将用于在学习准确性和学习成本（或学习时间）之间取得适当的平衡。该项目由电气，通信和网络系统（ECC）和既定计划的计划共同资助，以刺激竞争激烈的计划研究（EPSCOR）。该奖项反映了NSF的法定任务，并被认为是通过基金会的知识分子优点和更广泛的影响审查标准来评估值得支持的。