CAREER: Foundations of Federated Multi-Task Learning

职业：联合多任务学习的基础

基本信息

批准号：
2145670
负责人：
Virginia Smith
金额：
$ 59.72万
依托单位：
Carnegie-Mellon University
依托单位国家：
美国
项目类别：
Continuing Grant
财政年份：
2022
资助国家：
美国
起止时间：
2022-06-01 至 2027-05-31
项目状态：
未结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2145670&HistoricalAwards=false
关键词：
CAREER Foundations Federated Multi Task

项目摘要

Mobile phones, wearable devices, and smart homes form just a few of the modern distributed networks generating a wealth of data each day. Due to the growing computational power of edge devices, coupled with concerns over transmitting private data, it is increasingly attractive to store data locally and push network computation to the edge. Federated learning explores training machine learning models at the edge in distributed networks. While federated learning has shown tremendous promise for enabling edge applications, practical deployment is currently stymied by a number of competing constraints. In addition to being accurate, federated learning methods must scale to potentially massive networks of devices, and must exhibit trustworthy behavior---addressing pragmatic concerns related to issues such as user privacy, fairness, and robustness. In this project, we explore multi-task learning, a technique that learns separate but related models for each device in the network, as a unified approach to address the competing constraints of federated learning. The objective of the project is to develop scalable multi-task learning methods that are suitable for practical federated networks, and to rigorously study the foundational properties of federated multi-task learning in terms of the goals of accuracy, scalability, and trustworthiness. In doing so, the research will unlock a new generation of federated learning systems that can holistically address the constraints of realistic federated networks.The goal of this project is to establish and rigorously study the use of federated multi-task learning. While the accuracy benefits of federated multi-task learning are well-known, the work charts two new directions. First, the project develops methods to realize multi-task learning at scale in massive federated networks. Secondly, the project shows that multi-task learning, by improving privacy, fairness, and robustness, is in fact key for trustworthy federated learning. The technical aims of the project work are divided into three thrusts. First, by approximating standard notions of multi-task learning, the project will develop and rigorously study a family of highly scalable federated multi-task learning objectives. Second, the privacy implications of multi-task learning will be analyzed and evaluated in order to understand trade-offs between privacy and utility in federated networks. Finally, this project will explore tensions between fairness (in terms of performance disparities across devices) and robustness (to data and model poisoning attacks) in federated learning. Although these goals may be at odds, this project aims to show that multi-task learning can inherently improve both fairness and robustness, helping both to be achieved jointly. Taken together, this work has the potential to cause a paradigm-shift in the way federated learning systems are designed, implemented, and analyzed.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

手机、可穿戴设备和智能家居只是每天产生大量数据的现代分布式网络中的一小部分。由于边缘设备的计算能力不断增强，加上对传输私有数据的担忧，本地存储数据并将网络计算推到边缘越来越有吸引力。联邦学习探索在分布式网络边缘训练机器学习模型。虽然联邦学习在支持边缘应用程序方面显示出巨大的前景，但实际部署目前受到许多竞争性限制的阻碍。除了准确之外，联邦学习方法还必须扩展到潜在的大规模设备网络，并且必须表现出值得信赖的行为——解决与用户隐私、公平性和鲁棒性等问题相关的务实问题。在这个项目中，我们探索多任务学习，这是一种为网络中的每个设备学习单独但相关的模型的技术，作为解决联邦学习的竞争约束的统一方法。该项目的目标是开发适用于实际联邦网络的可扩展多任务学习方法，并严格研究联邦多任务学习在准确性、可扩展性和可信性目标方面的基本属性。在此过程中，该研究将解锁新一代联邦学习系统，该系统可以全面解决现实联邦网络的限制。该项目的目标是建立并严格研究联邦多任务学习的使用。虽然联合多任务学习的准确性优势是众所周知的，但该工作描绘了两个新的方向。首先，该项目开发了在大规模联邦网络中实现大规模多任务学习的方法。其次，该项目表明，通过提高隐私性、公平性和鲁棒性，多任务学习实际上是值得信赖的联邦学习的关键。该项目工作的技术目标分为三个重点。首先，通过近似多任务学习的标准概念，该项目将开发并严格研究一系列高度可扩展的联合多任务学习目标。其次，将分析和评估多任务学习的隐私影响，以了解联邦网络中隐私和实用性之间的权衡。最后，该项目将探讨联邦学习中公平性（就跨设备的性能差异而言）和鲁棒性（针对数据和模型中毒攻击）之间的紧张关系。尽管这些目标可能不一致，但该项目旨在表明多任务学习可以本质上提高公平性和鲁棒性，帮助两者共同实现。总而言之，这项工作有可能导致联邦学习系统的设计、实施和分析方式发生范式转变。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力优势和更广泛的评估进行评估，被认为值得支持。影响审查标准。