CRII: III: Towards Improving the Handling of Heterogeneity and Personalization in Federated Learning

CRII：III：改进联邦学习中异构性和个性化的处理

• 批准号: 2246067
• 负责人: Lichao Sun
• 金额: $ 17.5万
• 依托单位: Lehigh University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-08-01 至 2025-07-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2246067&HistoricalAwards=false
• 关键词: CRII; III; Towards; Improving; Handling

As awareness of the need for privacy preservation continues to grow in society, new legal restrictions, such as the General Data Protection Regulation (GDPR), are emerging. Such laws demand that businesses and organizations not share their clients' raw data for any commercial purposes. Federated Learning (FL) is a distributed machine learning paradigm that works with decentralized data while preserving privacy. FL has gained widespread interest and has been applied in numerous applications, such as healthcare, education, and intelligent manufacturing. However, FL faces some challenges that come from, executing on diverse types of data and devices, such as mobile phones and Internet of Things (IoT) devices. This project aims to address the aforementioned issues in heterogeneous FL by developing mathematical models and efficient algorithms. In addition, the project will integrate trustworthy ML research into new curriculum development and support students from underrepresented groups.Heterogeneous FL faces two significant challenges: (1) each client in FL may generate data according to a distinct distribution; (2) heterogeneous clients, such as mobile phones and IoT devices, are equipped with a wide range of computation and communication capabilities. To address these challenges, this project will dramatically push the boundary of knowledge via the following two integrated research thrusts: (i) The research team aims to tackle data heterogeneity in FL by designing two advanced personalized learning methods. Specifically, the proposed solutions aim to balance the generalization ability from the global model and the personalization ability from the local model, improving both the global model and personalized local models. (ii) The team will study heterogeneous neural network aggregation for FL by providing advanced memory-efficient local training strategies for small devices. In addition, the project will make use of mutual knowledge distillation to improve the generalization ability of the local models. Finally, the team proposes a unified FL framework that integrates data-free knowledge aggregation with advanced memory-efficient solutions to tackle both heterogeneity issues simultaneously.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

随着社会对隐私保护必要性的认识不断增强，新的法律限制不断涌现，例如《通用数据保护条例》(GDPR)。此类法律要求企业和组织不得出于任何商业目的共享客户的原始数据。联合学习 (FL) 是一种分布式机器学习范例，可在保护隐私的同时处理分散数据。 FL 引起了广泛的兴趣，并已应用于医疗、教育、智能制造等众多领域。然而，FL 面临着一些挑战，这些挑战来自于在不同类型的数据和设备上执行，例如移动电话和物联网 (IoT) 设备。该项目旨在通过开发数学模型和高效算法来解决异构 FL 中的上述问题。此外，该项目还将把值得信赖的机器学习研究融入到新课程开发中，并为来自代表性不足群体的学生提供支持。异构 FL 面临两个重大挑战：（1）每个 FL 客户可能会根据不同的分布生成数据； (2)异构客户端，例如手机和物联网设备，配备了广泛的计算和通信能力。为了应对这些挑战，该项目将通过以下两个综合研究重点极大地拓展知识的边界：（i）研究团队旨在通过设计两种先进的个性化学习方法来解决 FL 中的数据异构性。具体来说，所提出的解决方案旨在平衡全局模型的泛化能力和局部模型的个性化能力，同时改进全局模型和个性化局部模型。 (ii) 该团队将通过为小型设备提供先进的内存高效本地训练策略来研究 FL 的异构神经网络聚合。此外，该项目将利用相互知识蒸馏来提高局部模型的泛化能力。最后，该团队提出了一个统一的 FL 框架，将无数据的知识聚合与先进的内存高效解决方案相结合，以同时解决异构性问题。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力优势和能力进行评估，被认为值得支持。更广泛的影响审查标准。