Federated Optimization over Bandwidth-Limited Heterogeneous Networks

带宽受限异构网络的联合优化

• 批准号: 2318441
• 负责人: Yuejie Chi
• 金额: $ 36万
• 依托单位: Carnegie-Mellon University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-09-01 至 2026-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2318441&HistoricalAwards=false
• 关键词: Federated; Optimization; over; Bandwidth; Limited

Harnessing the power of data collected from a vast amount of geographically distributed and heterogeneous devices, in a manner without moving data around and violating privacy, has great potential in advancing science and technology and improving quality of life. Federated optimization lies at the heart of the practice realizing this vision, encompassing problems such as training large-scale machine learning or artificial intelligence models, delivering insightful data analytics, as well as facilitating decision making under uncertainty, all in distributed manners. There is a significant gap in the algorithmic foundation of federated optimization when interfacing with bandwidth-limited heterogeneous networks, such as internet-of-things, smart healthcare, and edge computing, to meet the unique challenges of taming heterogeneity, privacy, and uncertainty without sacrificing efficiency. This research project will also be tightly integrated with education and workforce developments, through offering new courses, mentoring students at all levels in research projects including underrepresented minorities and women, and disseminating the research outcomes at suitable conferences and workshops. The goal of the research program is to develop a federated optimization framework to learning and decision making by designing communication-efficient, computation-scalable, and privacy-preserving algorithms that converge provably over highly heterogeneous data and computing environments. Leveraging insights from machine learning, optimization theory, signal processing, and differential privacy, the research program offers an entirely new suite of theoretical and algorithmic tools to enable heterogeneity-embracing and privacy-preserving learning and decision making in federated environments under bandwidth constraints, unveiling fundamental trade-offs among computation, communication, privacy, and utility. The research program will gravitate around a semi-decentralized federated setting suitable to meet the diverse needs of bandwidth-limited heterogeneous networks, and focus on developing bandwidth-limited federated optimization algorithms that are efficient, resilient, and private with rigorous performance guarantees for a wide range of problems arising from machine learning, data analysis, and sequential decision making.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

以不移动数据和侵犯隐私的方式利用从大量地理分布的异构设备收集的数据的力量，在推动科学技术和提高生活质量方面具有巨大潜力。联合优化是实现这一愿景的实践的核心，包括训练大规模机器学习或人工智能模型、提供有洞察力的数据分析以及促进不确定性下的决策等问题，所有这些都以分布式方式进行。当与物联网、智能医疗和边缘计算等带宽有限的异构网络接口时，联邦优化的算法基础存在显着差距，无法满足驾驭异构性、隐私和不确定性的独特挑战。牺牲效率。该研究项目还将通过提供新课程、指导各级学生参与研究项目（包括代表性不足的少数群体和妇女）以及在适当的会议和研讨会上传播研究成果，与教育和劳动力发展紧密结合。该研究计划的目标是通过设计通信高效、计算可扩展和隐私保护的算法来开发学习和决策的联合优化框架，这些算法可在高度异构的数据和计算环境中证明收敛。该研究项目利用机器学习、优化理论、信号处理和差异隐私的见解，提供了一套全新的理论和算法工具，以在带宽限制的联邦环境中实现异构性拥抱和隐私保护的学习和决策，揭示了计算、通信、隐私和效用之间的基本权衡。该研究计划将围绕适合满足带宽有限异构网络的多样化需求的半去中心化联邦设置，并专注于开发高效、有弹性和私密性的带宽有限联邦优化算法，并为广泛的应用提供严格的性能保证。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Escaping Saddle Points in Heterogeneous Federated Learning via Distributed SGD with Communication Compression

通过分布式 SGD 和通信压缩摆脱异构联邦学习中的鞍点

• 发表时间: 2024-05
• 期刊: Proceedings of the 27th International Conference on Artificial Intelligence and Statistics (AISTATS
• 作者: Chen, Sijin;Li, Zhize;Chi, Yuejie
• 通讯作者: Chi, Yuejie