Collaborative Research: SaTC: CORE: Small: Foundations for the Next Generation of Private Learning Systems

协作研究：SaTC：核心：小型：下一代私人学习系统的基础

• 批准号: 2120667
• 负责人: Adam Smith
• 金额: $ 10万
• 依托单位: Trustees of Boston University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-10-01 至 2023-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2120667&HistoricalAwards=false
• 关键词: Collaborative; Research; SaTC; CORE; Small

Recent advances in large-scale machine learning (ML) promise a range of benefits to society, but also introduce new risks. One major risk is a loss of privacy for the individuals whose data powers the machine learning algorithms. There are now convincing demonstrations that algorithms for machine learning can reveal sensitive information about individuals in their training data by memorizing specific strings of sensitive text such as bank account numbers or through membership-inference attacks. In the recent years, a framework called differential privacy---a mathematically principled, quantitative notion of what it means for an algorithm to ensure privacy for the individuals who contribute training data---has led to significant progress towards privacy in machine learning. This progress offers a proof-of-concept that we can hope to enjoy some of the benefits of using machine learning on sensitive data, while measuring and limiting breaches of confidentiality. This project will investigate and begin to make some of the fundamental advances that are necessary to make differentially private ML a viable technology. The focus will be on laying the groundwork for differentially private ML for entire systems, rather than for standalone tasks, which have been the focus of prior work. This project team comprising researchers with a broad range of expertise in ML, algorithms, systems, and cybersecurity, has planned a set of education tasks: public-facing set of course materials on differentially private machine learning and statistics and and an undergraduate-level textbook on differential privacy.This project includes three technical thrusts that will lay the groundwork for future efforts to build private ML systems. The first thrust will be to improve the foundational algorithms that enable differentially private ML on high-dimensional data. The second thrust will be to build a bridge between algorithms for standalone ML tasks and algorithms for systems-level workloads of ML tasks, by developing differentially private algorithms for training many personalized models, which is a paradigmatic workload in ML. The final thrust will consist of empirical work on auditing differentially private ML methods to understand how the real-world privacy costs compare to those predicted by the theory of differential privacy when these algorithms are used as part of realistic workloads, such as models that are continually updated with new data. This privacy auditing will also facilitate detecting unwanted memorization of training data in machine learning, and also provide more quantitative approaches to auditing differentially private algorithms based on membership-inference and data poisoning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

大规模机器学习 (ML) 的最新进展有望为社会带来一系列好处，但也带来了新的风险。 一项主要风险是为机器学习算法提供数据支持的个人的隐私丧失。现在有令人信服的证据表明，机器学习算法可以通过记忆特定的敏感文本字符串（例如银行帐号）或通过成员推理攻击来揭示训练数据中有关个人的敏感信息。近年来，一种称为差异隐私的框架（一种数学原理的定量概念，说明算法对于确保贡献训练数据的个人的隐私意味着什么）已经在机器学习的隐私方面取得了重大进展。这一进展提供了一个概念验证，即我们希望能够享受到在敏感数据上使用机器学习的一些好处，同时衡量和限制机密性的泄露。 该项目将调查并开始取得一些必要的基本进展，使差分隐私机器学习成为可行的技术。 重点将是为整个系统的差异化私有机器学习奠定基础，而不是为独立任务奠定基础，后者是之前工作的重点。 该项目团队由在机器学习、算法、系统和网络安全领域拥有广泛专业知识的研究人员组成，计划了一系列教育任务：面向公众的一套关于差异化私有机器学习和统计的课程材料以及一本本科水平的教科书该项目包括三个技术要点，将为未来构建私有机器学习系统奠定基础。 第一个目标是改进基础算法，以实现高维数据上的差分隐私机器学习。 第二个重点是通过开发用于训练许多个性化模型（ML 中的典型工作负载）的差分私有算法，在独立 ML 任务的算法和 ML 任务的系统级工作负载算法之间建立一座桥梁。 最后的重点将包括审核差分隐私 ML 方法的实证工作，以了解当这些算法用作实际工作负载的一部分（例如持续不断的模型）时，现实世界的隐私成本与差分隐私理论预测的隐私成本相比如何。更新了新数据。这种隐私审计还将有助于检测机器学习中不必要的训练数据记忆，并提供更多定量方法来审计基于成员推理和数据中毒的差分隐私算法。该奖项反映了 NSF 的法定使命，并被认为值得通过以下方式获得支持：使用基金会的智力价值和更广泛的影响审查标准进行评估。

项目成果

Improved Differential Privacy for SGD via Optimal Private Linear Operators on Adaptive Streams

• 发表时间: 2022-02
• 作者: S. Denisov;H. B. McMahan;J. Rush;Adam D. Smith;Abhradeep Thakurta

Fast, Sample-Efficient, Affine-Invariant Private Mean and Covariance Estimation for Subgaussian DistributionsGavin Brown and Samuel B. Hopkins and Adam D. Smith

亚高斯分布的快速、样本高效、仿射不变私有均值和协方差估计Gavin Brown、Samuel B. Hopkins 和 Adam D. Smith

• 发表时间: 2023
• 期刊: COLT 2023
• 作者: Brown, Gavin;Hopkins, Samuel B;Smith, Adam D
• 通讯作者: Smith, Adam D

Strong Memory Lower Bounds for Learning Natural Models

• DOI: 10.48550/arxiv.2206.04743
• 发表时间: 2022-06
• 期刊: ArXiv
• 作者: Gavin Brown;Mark Bun;Adam M. Smith

Counting Distinct Elements in the Turnstile Model with Differential Privacy under Continual Observation

在持续观察下计算具有差异隐私的旋转栅门模型中的不同元素

• 发表时间: 2023
• 期刊: NeurIPS 2023
• 作者: Kalemaj, Iden;Jain, Palak;Raskhodnikova, Sofya;Sivakumar, Satchit;Smith, Adam D
• 通讯作者: Smith, Adam D

The Price of Differential Privacy under Continual Observation

• 发表时间: 2021-12
• 作者: Palak Jain;Sofya Raskhodnikova;Satchit Sivakumar;Adam D. Smith