RI: Small: Secure, Private, and Resource-Constrained Approaches to Federated Machine Learning

RI：小型：安全、私有且资源受限的联合机器学习方法

• 批准号: 1909577
• 负责人: Oluwasanmi Koyejo
• 金额: $ 45万
• 依托单位: University of Illinois at Urbana-Champaign
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-08-01 至 2022-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1909577&HistoricalAwards=false
• 关键词: RI; Small; Secure; Private; Resource

In a world increasingly shaped by data-driven machine learning (ML), one of the emerging challenges is that data are often collected and stored in a distributed manner -- across multiple datacenters or devices. On the other hand, due to security and privacy concerns, there are often low levels of trust between the data owners. To this end, federated ML enables ML with distributed data, while avoiding the transfer of private data from distributed devices to a central datacenter. Towards the goal of democratizing ML, this project will design and implement new techniques to make federated ML secure and private. Of particular interest are new system designs that enable federated ML on devices with limited computational power or communication bandwidth e.g., smartphones, smart health monitors, and smartwatches, among others. The ideas, software, and results of this project will directly impact industry and real-world applications. This project will include curriculum development for federated ML and plans to involve participation by graduate students from underrepresented groups. This project creates a transformative new direction for federated machine learning (ML) research, by enabling ML on devices that are untrusted or weak, and across organizations and for users who would like to maintain the privacy of their data. This project will include new work on theoretical foundations, systems design, implementation, and integration with popular ML software. Concretely, this project tackles three challenges in federated ML. The first challenge is fault-tolerant ML algorithms, i.e., new techniques to perform ML when workers act in arbitrarily malicious manners (called Byzantine failures) -- in particular, this project will show that by leveraging natural noise-tolerance in ML, it is possible to tolerate significantly more Byzantine workers than indicated by the traditional distributed computing literature. The second challenge is to develop privacy-preserving ML algorithms which introduce noise from workers to preserve the privacy of data owned by participants while leading to correct and fast ML at the global level. The third challenge is to investigate resource-constrained ML scheduling by including new techniques to allow large neural network models to run across multiple devices which have memory constraints. In addition to developing the algorithmic and theoretical frameworks for these directions, this project will also build and release open software.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在一个越来越多地受数据驱动的机器学习（ML）塑造的世界中，新出现的挑战之一是，在多个数据中心或设备之间，通常以分布式方式收集和存储数据。另一方面，由于安全性和隐私问题，数据所有者之间的信任程度通常很低。为此，联合ML可以使用分布式数据启用ML，同时避免将私人数据从分布式设备传输到中央数据中心。为了使ML民主化，该项目将设计和实施新技术，以使联盟的ML安全和私密。特别令人感兴趣的是新的系统设计，可以在具有有限的计算能力或通信带宽的设备上进行联合ML，例如智能手机，智能健康监控器和智能手表等。该项目的想法，软件和结果将直接影响行业和现实世界应用。该项目将包括针对联合ML的课程开发，并计划涉及来自代表性不足小组的研究生的参与。该项目通过在不信任或弱的设备上以及跨组织以及希望维护其数据隐私的用户的设备上启用ML来为联合机器学习（ML）研究创建一个变革性的新方向。该项目将包括有关理论基础，系统设计，实现和与流行ML软件集成的新工作。具体而言，该项目在联邦ML中解决了三个挑战。第一个挑战是容忍失误的ML算法，即当工人以任意恶意的方式行事（称为拜占庭失败）时，可以执行ML的新技术 - 特别是，该项目将表明，通过在ML中利用自然的噪音耐受性，可以通过传统的计算文献来容忍更大的拜占庭工人。第二个挑战是开发保护隐私的ML算法，该算法引入工人的噪音，以保留参与者拥有的数据的隐私，同时导致全球层面上的正确和快速ML。第三个挑战是通过包括新技术来允许大型神经网络模型跨具有内存约束的多个设备运行，以研究资源受限的ML调度。除了为这些方向开发算法和理论框架外，该项目还将构建和发布开放软件。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子和更广泛影响的评估审查标准通过评估来支持的。

项目成果

Adversarially Robust Models may not Transfer Better: Sufficient Conditions for Domain Transferability from the View of Regularization

• 发表时间: 2022-02
• 作者: Xiaojun Xu;Jacky Y. Zhang;Evelyn Ma;Danny Son;Oluwasanmi Koyejo;Bo Li

ZenoPS: A Distributed Learning System Integrating Communication Efficiency and Security

ZenoPS：集通信效率与安全于一体的分布式学习系统

• DOI: 10.3390/a15070233
• 发表时间: 2022
• 期刊: Algorithms
• 影响因子: 2.3
• 作者: Xie, Cong;Koyejo, Oluwasanmi;Gupta, Indranil
• 通讯作者: Gupta, Indranil

EMIXER: End-to-end Multimodal X-ray Generation via Self-supervision

• 发表时间: 2020-07
• 期刊: ArXiv
• 作者: Siddharth Biswal;Peiye Zhuang;A. Pyrros;Nasir Siddiqui;Oluwasanmi Koyejo;Jimeng Sun

Quadratic metric elicitation for fairness and beyond

• 发表时间: 2020-11
• 作者: G. Hiranandani;Jatin Mathur;H. Narasimhan;Oluwasanmi Koyejo

A Word is Worth A Thousand Dollars: Adversarial Attack on Tweets Fools Stock Prediction

• DOI: 10.48550/arxiv.2205.01094
• 发表时间: 2022-05
• 作者: Yong Xie;Dakuo Wang;Pin-Yu Chen;Jinjun Xiong;Sijia Liu;Oluwasanmi Koyejo