RI: Small: Secure, Private, and Resource-Constrained Approaches to Federated Machine Learning

RI：小型：安全、私有且资源受限的联合机器学习方法

• 批准号: 1909577
• 负责人: Oluwasanmi Koyejo
• 金额: $ 45万
• 依托单位: University of Illinois at Urbana-Champaign
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-08-01 至 2022-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1909577&HistoricalAwards=false
• 关键词: RI; Small; Secure; Private; Resource

In a world increasingly shaped by data-driven machine learning (ML), one of the emerging challenges is that data are often collected and stored in a distributed manner -- across multiple datacenters or devices. On the other hand, due to security and privacy concerns, there are often low levels of trust between the data owners. To this end, federated ML enables ML with distributed data, while avoiding the transfer of private data from distributed devices to a central datacenter. Towards the goal of democratizing ML, this project will design and implement new techniques to make federated ML secure and private. Of particular interest are new system designs that enable federated ML on devices with limited computational power or communication bandwidth e.g., smartphones, smart health monitors, and smartwatches, among others. The ideas, software, and results of this project will directly impact industry and real-world applications. This project will include curriculum development for federated ML and plans to involve participation by graduate students from underrepresented groups. This project creates a transformative new direction for federated machine learning (ML) research, by enabling ML on devices that are untrusted or weak, and across organizations and for users who would like to maintain the privacy of their data. This project will include new work on theoretical foundations, systems design, implementation, and integration with popular ML software. Concretely, this project tackles three challenges in federated ML. The first challenge is fault-tolerant ML algorithms, i.e., new techniques to perform ML when workers act in arbitrarily malicious manners (called Byzantine failures) -- in particular, this project will show that by leveraging natural noise-tolerance in ML, it is possible to tolerate significantly more Byzantine workers than indicated by the traditional distributed computing literature. The second challenge is to develop privacy-preserving ML algorithms which introduce noise from workers to preserve the privacy of data owned by participants while leading to correct and fast ML at the global level. The third challenge is to investigate resource-constrained ML scheduling by including new techniques to allow large neural network models to run across multiple devices which have memory constraints. In addition to developing the algorithmic and theoretical frameworks for these directions, this project will also build and release open software.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

在数据驱动的机器学习 (ML) 日益影响的世界中，新出现的挑战之一是数据通常以分布式方式收集和存储 - 跨多个数据中心或设备。另一方面，出于安全和隐私方面的考虑，数据所有者之间的信任度往往较低。为此，联合机器学习支持使用分布式数据进行机器学习，同时避免将私有数据从分布式设备传输到中央数据中心。为了实现 ML 民主化的目标，该项目将设计和实施新技术，使联合 ML 安全且私密。特别令人感兴趣的是新的系统设计，可以在计算能力或通信带宽有限的设备（例如智能手机、智能健康监视器和智能手表等）上实现联合机器学习。该项目的想法、软件和结果将直接影响行业和现实世界的应用。该项目将包括联合机器学习的课程开发，并计划让来自代表性不足群体的研究生参与。该项目通过在不受信任或薄弱的设备上、跨组织以及希望维护数据隐私的用户启用机器学习，为联合机器学习 (ML) 研究创造了一个变革性的新方向。该项目将包括理论基础、系统设计、实现以及与流行的机器学习软件集成方面的新工作。具体来说，该项目解决了联合机器学习中的三个挑战。第一个挑战是容错 ML 算法，即当工作人员以任意恶意方式行事（称为拜占庭故障）时执行 ML 的新技术 - 特别是，该项目将表明，通过利用 ML 中的自然噪声容忍度，可以容忍比传统分布式计算文献显示的更多的拜占庭工人。第二个挑战是开发保护隐私的机器学习算法，该算法会引入工作人员的噪音，以保护参与者拥有的数据的隐私，同时在全球范围内实现正确、快速的机器学习。第三个挑战是通过采用新技术来研究资源受限的机器学习调度，以允许大型神经网络模型在具有内存限制的多个设备上运行。除了开发这些方向的算法和理论框架外，该项目还将构建和发布开放软件。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

ZenoPS: A Distributed Learning System Integrating Communication Efficiency and Security

ZenoPS：集通信效率与安全于一体的分布式学习系统

• DOI: 10.3390/a15070233
• 发表时间: 2022-07
• 期刊: Algorithms
• 影响因子: 2.3
• 作者: Xie, Cong;Koyejo, Oluwasanmi;Gupta, Indranil
• 通讯作者: Gupta, Indranil

EMIXER: End-to-end Multimodal X-ray Generation via Self-supervision

EMIXER：通过自我监督生成端到端多模态 X 射线

• DOI: 10.1016/j.jphotobiol.2010.06.017
• 发表时间: 2020-07-10
• 期刊: ArXiv
• 作者: Siddharth Biswal;Peiye Zhuang;A. Pyrros;Nasir Siddiqui;Oluwasanmi Koyejo;Jimeng Sun
• 通讯作者: Jimeng Sun

Adversarially Robust Models may not Transfer Better: Sufficient Conditions for Domain Transferability from the View of Regularization

对抗性鲁棒模型可能无法更好地迁移：从正则化的角度来看域可迁移性的充分条件

• 发表时间: 2022-02-03
• 作者: Xiaojun Xu;Jacky Y. Zhang;Evelyn Ma;Danny Son;Oluwasanmi Koyejo;Bo Li
• 通讯作者: Bo Li

A Nonconvex Framework for Structured Dynamic Covariance Recovery}

结构化动态协方差恢复的非凸框架}

• 发表时间: 2022-01
• 期刊: Journal of machine learning research
• 影响因子: 6
• 作者: Tsai, Katherine;Kolar, Mladen;Koyejo, Oluwasanmi
• 通讯作者: Koyejo, Oluwasanmi

A Word is Worth A Thousand Dollars: Adversarial Attack on Tweets Fools Stock Prediction

一个词值一千美元：对推文的对抗性攻击愚弄了股票预测

• DOI: 10.48550/arxiv.2205.01094
• 发表时间: 2022-05-01
• 作者: Yong Xie;Dakuo Wang;Pin;Jinjun Xiong;Sijia Liu;Oluwasanmi Koyejo
• 通讯作者: Oluwasanmi Koyejo