AF: Small: Collaborative Research: Rigorous Approaches for Scalable Privacy-preserving Deep Learning

AF：小型：协作研究：可扩展的隐私保护深度学习的严格方法

基本信息

批准号：
1908384
负责人：
C Sesh Seshadhri
金额：
$ 8.19万
依托单位：
University of California-Santa Cruz
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2019
资助国家：
美国
起止时间：
2019-10-01 至 2023-09-30
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1908384&HistoricalAwards=false
关键词：
AF Small Collaborative Research Rigorous

项目摘要

One of the most salient features of this time is the dissemination of massive amounts of personal and sensitive data. Despite their enormous societal benefits, the powerful tools of modern machine learning, especially deep learning, can pose real threats to personal privacy. For example, over the last few years, it has become evident that deep neural networks have a remarkable power in learning even the finest details from large complex data sets. With such powerful tools, the need for robust and rigorous guarantees for privacy protection has become more crucial. The last decade has witnessed the rise of a sound mathematical theory, known as differential privacy, that enables designing data-analysis algorithms with rigorous privacy guarantees for their input data sets. Despite the noticeable success of this theory, existing tools from differential privacy are severely limited in offering acceptable utility guarantees when dealing with complex models like those arising in deep learning. This project will address those limitations by offering new principled approaches for designing differentially-private deep-learning algorithms that can scale to industrial workloads. The project will also involve collaboration with industry, which will facilitate the evaluation of the developed algorithms on real-world datasets and the development of open-source software tools. The products of this project have the potential to transform the way massive sets of sensitive data are used in modern machine-learning systems, which will impact the way these systems are designed and implemented in practice. The activities of this project will also aim at promoting diversity in computing by recruiting women and members of underrepresented groups.The investigators will develop a rigorous, multi-faceted design paradigm for scalable, practical, differentially private algorithms for modern machine learning. This paradigm is based on two general strategies: (i) exploiting realistic and useful properties of the data and the machine-learning models to circumvent existing limitations in the literature on differential privacy, and (ii) leveraging a limited amount of public data (that has no privacy constraints) to boost the utility of the algorithms. Based on these strategies, the project will pursue following directions: (1) developing a new, generic framework for utilizing public data in privacy-preserving machine learning, (2) designing improved iterative training algorithms that can bypass the standard use of the so-called "composition theorem" of differential privacy, and (3) designing new differentially private stochastic-gradient methods tuned specifically to non-convex and over-parameterized machine-learning problems.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

这段时间最突出的特征之一是传播大量个人和敏感数据。尽管具有巨大的社会利益，但现代机器学习的强大工具，尤其是深度学习，可以对个人隐私构成真正的威胁。例如，在过去的几年中，很明显，深度神经网络甚至可以从大型复杂数据集中学习最好的细节也具有显着的力量。借助如此强大的工具，对保护隐私保护的强大和严格保证的需求变得更加至关重要。在过去的十年中，见证了一个合理的数学理论（称为差异隐私）的兴起，该理论可以为其输入数据集设计具有严格隐私保证的数据分析算法。尽管该理论取得了显着的成功，但在处理像深度学习中出现的复杂模型时，现有的差异隐私工具在提供可接受的效用保证方面受到严重限制。该项目将通过提供新的原则方法来设计差异性深度学习算法来解决这些限制，以扩展到工业工作负载。该项目还将涉及与行业的合作，这将促进对现实数据集中开发算法的评估以及开发开源软件工具的开发。该项目的产品有可能改变现代机器学习系统中大量敏感数据集的方式，这将影响这些系统在实践中设计和实施的方式。该项目的活动还将旨在通过招募妇女和代表性不足的群体的成员来促进计算中的多样性。研究人员将开发一种严格的，多方面的设计范式，用于可扩展，实用，差异化的私人算法，用于现代机器学习。该范式基于两种一般策略：（i）利用数据的现实且有用的特性和机器学习模型来规避文献中有关差异隐私的现有限制，以及（ii）利用有限的公共数据（没有隐私限制）来提高算法的实用性。基于这些策略，该项目将追求以下方向：（1）开发一个新的通用框架，用于在隐私保护机器学习中利用公共数据，（2）设计改进的迭代培训算法，可以绕过所谓的“组成定理”的标准用法，以设计差异性私密和（3）设计的“组成”，以及（3）设计不合格的私人私有方法，以及（3），并（3）设计不合格的方法，并（3）过度参数化的机器学习问题。该奖项反映了NSF的法定任务，并通过使用基金会的知识分子优点和更广泛的影响审查标准来评估值得支持。