MLWiNS: Democratizing AI through Multi-Hop Federated Learning Over-the-Air

MLWiNS：通过多跳联合无线学习使人工智能民主化

• 批准号: 2003198
• 负责人: Pu Wang
• 金额: $ 44.67万
• 依托单位: University of North Carolina at Charlotte
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-07-01 至 2024-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2003198&HistoricalAwards=false
• 关键词: MLWiNS; Democratizing; AI; through; Multi

Federated learning (FL) has emerged as a key technology for enabling next-generation privacy-preserving AI at-scale, where a large number of edge devices, e.g., mobile phones, collaboratively learn a shared global model while keeping their data locally to prevent privacy leakage. Enabling FL over wireless multi-hop networks, such as wireless community mesh networks and wireless Internet over satellite constellations, not only can augment AI experiences for urban mobile users, but also can democratize AI and make it accessible in a low-cost manner to everyone, including people in low-income communities, rural areas, under-developed regions, and disaster areas. The overall objective of this project is to develop a novel wireless multi-hop FL system with guaranteed stability, high accuracy and fast convergence speed. This project is expected to advance the design of distributed deep learning (DL) systems, to promote the understanding of the strong synergy between distributed computing and distributed networking, and to bridge the gap between the theoretical foundations of distributed DL and its real-life applications. The project will also provide unique interdisciplinary training opportunities for graduate and undergraduate students through both research work and related courses that the PIs will develop and offer. This project proposes to use concepts of federated learning and multi-agent reinforcement learning to provide optimal solutions for training DL models over wireless multi-hop networks that have communication constraints due to noisy and interference-rich wireless links. The main thrusts include: 1) developing a novel hierarchical FL system architecture with layered federated computation, semi-asynchronous model aggregation, and regularized objective function to significantly improve system scalability, communication efficiency, and stability; 2) fine-tuning the FL system via multi-agent reinforcement learning to maximize the FL accuracy with the minimum convergence time under the computing constraints of edge devices; 3) finding high-gain computation-light robust federated computing strategies for resource-constraint edge devices, including efficient DL model design and resource-aware model adaptation; and 4) developing an open-source wireless FL framework (OpenWFL) for fast prototyping, deploying, and evaluating the proposed FL algorithms in both an emulator and physical testbeds.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Federated学习（FL）已成为启用下一代隐私性AI AI ATCALE的关键技术，其中大量的边缘设备（例如移动电话）协作地学习了共享的全球模型，同时在本地保留其数据以防止隐私泄漏。通过无线多跳网络启用FL，例如无线社区网络网络和卫星星座上的无线互联网，不仅可以增强城市移动用户的AI体验，而且还可以使AI民主化并使其以低成本的方式使每个人都可以访问，包括低收入社区，农村地区，不足的地区，不断发展的地区和灾难地区和灾难。该项目的总体目的是开发一种具有保证稳定性，高精度和快速收敛速度的新型无线多跳力系统。预计该项目将推进分布式深度学习（DL）系统的设计，以促进对分布式计算和分布式网络之间强大协同作用的理解，并弥合分布式DL的理论基础及其现实生活中的理论基础之间的差距。该项目还将通过PIS将开发和提供的研究工作和相关课程为研究生和本科生提供独特的跨学科培训机会。 该项目建议使用联合学习和多机构强化学习的概念，为无线多跳网络的培训DL模型提供最佳解决方案，这些网络由于嘈杂和干扰富含无线的无线链接而具有通信约束。主要推力包括：1）开发具有分层联合计算，半同步模型聚合和正则目标函数的新型分层FL系统体系结构，以显着提高系统的可扩展性，通信效率和稳定性； 2）通过多代理增强学习对FL系统进行微调，以在边缘设备的计算限制下使用最小收敛时间最大化FL精度； 3）寻找资源构成边缘设备的高增益计算稳健的联合计算策略，包括有效的DL模型设计和资源感知模型适应； 4）开发一个开源无线FL框架（OpenWFL），用于快速原型制作，部署和评估模拟器和物理测试床中所提出的FL算法。这项奖项反映了NSF的法定任务，并被认为是通过基金会的智力优点和广泛的影响来评估CRETERIA的评估。

项目成果

MutualNet: Adaptive ConvNet via Mutual Learning From Different Model Configurations

• DOI: 10.1109/tpami.2021.3138389
• 发表时间: 2021-05
• 期刊: IEEE Transactions on Pattern Analysis and Machine Intelligence
• 影响因子: 23.6
• 作者: Taojiannan Yang;Sijie Zhu;Mat'ias Mendieta;Pu Wang;Ravikumar Balakrishnan;Minwoo Lee;T. Han

Local Learning Matters: Rethinking Data Heterogeneity in Federated Learning

• DOI: 10.1109/cvpr52688.2022.00821
• 发表时间: 2022-01-01
• 期刊: 2022 IEEE/CVF CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR)
• 作者: Mendieta, Matias;Yang, Taojiannan;Chen, Chen
• 通讯作者: Chen, Chen

Privacy Enhancement for Cloud-Based Few-Shot Learning

• DOI: 10.1109/ijcnn55064.2022.9892135
• 发表时间: 2022-05
• 期刊: 2022 International Joint Conference on Neural Networks (IJCNN)
• 作者: Archit Parnami;M. Usama;Liyue Fan;Minwoo Lee

Sim-to-Real Transfer in Multi-agent Reinforcement Networking for Federated Edge Computing

• DOI: 10.1145/3453142.3491419
• 发表时间: 2021-10
• 期刊: 2021 IEEE/ACM Symposium on Edge Computing (SEC)
• 作者: Pinyarash Pinyoanuntapong;Tagore Pothuneedi;Ravikumar Balakrishnan;Minwoo Lee;Chen Chen-Chen;Pu Wang

EdgeML: Towards Network-Accelerated Federated Learning over Wireless Edge

• DOI: 10.1016/j.comnet.2022.109396
• 发表时间: 2021-10
• 期刊: Comput. Networks
• 作者: Pinyarash Pinyoanuntapong;Prabhu Janakaraj;Ravikumar Balakrishnan;Minwoo Lee;Chen Chen-Chen;Pu Wang