CAREER: Robust Adaptive Optimization Algorithms for Differentially Private Learning

职业：用于差异化私人学习的鲁棒自适应优化算法

• 批准号: 1943046
• 负责人: Jaewoo Lee
• 金额: $ 52.96万
• 依托单位: University of Georgia Research Foundation Inc
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-03-01 至 2025-02-28
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1943046&HistoricalAwards=false
• 关键词: CAREER; Robust; Adaptive; Optimization; Algorithms

Privacy-preserving optimization algorithms are essential tools for solving machine learning (ML) problems while protecting the privacy of individuals in the datasets used for training ML models. Despite the recent advances, there is a lack of a theoretical foundation for understanding their performance and hence their use in practice is limited because of utility concerns. This project seeks to develop a theory to understand the performance of private optimizers and use it to guide the design of algorithms with reliable and robust performance. To this end, the project focuses on the three main challenges related to differentially private learning: (i) bridging the gap between theory and practice by developing a unified theoretical framework that can be used to better understand and explain the performance of private optimizers; (ii) applying the theory to guide the design of private optimizers whose privacy and utility guarantees have robustness to hyperparameter choices; (iii) extending the framework, established principles, and algorithms to deep learning models. The project’s novelty is in providing a unified theoretical framework that enables rigorous performance analysis of private optimizers. By providing a theoretical foundation, this project will help accelerate research on differentially private learning, for example, by allowing the principled design of robust and reliable training algorithms. More broadly, this project has a great potential to accelerate advances in other domains of science by providing tools to share and analyze sensitive data without having to sacrifice the privacy of individuals. The project involves both graduate and undergraduate students in this research.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

隐私保护优化算法是解决机器学习 (ML) 问题的重要工具，同时保护用于训练 ML 模型的数据集中的个人隐私，尽管最近取得了进展，但仍缺乏理解其性能的理论基础。由于实用性问题，它们在实践中的使用受到限制。该项目旨在开发一种理论来理解私有优化器的性能，并用它来指导具有可靠和稳健性能的算法的设计。与差别化私人学习相关的主要挑战：(i)通过开发一个统一的理论框架来弥合理论与实践之间的差距，该框架可用于更好地理解和解释私有优化器的性能；（ii）应用该理论来指导私有优化器的设计，其隐私和效用保证对超参数具有鲁棒性； (iii) 将框架、既定原理和算法扩展到深度学习模型。该项目的新颖之处在于提供了一个统一的理论框架，可以对私有优化器进行严格的性能分析。通过提供理论基础，该项目将有助于加速差异化研究。私人学习，例如，通过更广泛地允许稳健可靠的训练算法的原则性设计，该项目具有巨大的潜力，可以通过提供共享和分析敏感数据的工具而无需牺牲个人隐私，从而加速其他科学领域的进步。该项目涉及研究生和本科生。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

Stochastic Adaptive Line Search for Differentially Private Optimization

• DOI: 10.1109/bigdata50022.2020.9378011
• 发表时间: 2020-08
• 期刊: 2020 IEEE International Conference on Big Data (Big Data)
• 作者: Chen Chen-Chen;Jaewoo Lee

Differentially Private Goodness-of-Fit Tests for Continuous Variables

• DOI: 10.1016/j.ecosta.2021.09.007
• 发表时间: 2021-10
• 期刊: Econometrics and Statistics
• 影响因子: 1.9
• 作者: Seungwoo Kwak;Jeongyoun Ahn;Jaewoo Lee;Cheolwoo Park

Scaling up Differentially Private Deep Learning with Fast Per-Example Gradient Clipping

• DOI: 10.2478/popets-2021-0008
• 发表时间: 2020-09
• 期刊: Proceedings on Privacy Enhancing Technologies
• 作者: Jaewoo Lee;Daniel Kifer

Differentially Private Normalizing Flows for Synthetic Tabular Data Generation

• DOI: 10.1609/aaai.v36i7.20697
• 发表时间: 2022-06
• 作者: Jaewoo Lee;Minjung Kim-;Yonghyun Jeong;Youngmin Ro

Performance Testing for Cloud Computing with Dependent Data Bootstrapping

使用依赖数据引导的云计算性能测试

• DOI: 10.1109/ase51524.2021.9678687
• 发表时间: 2021
• 作者: He, Sen;Liu, Tianyi;Lama, Palden;Lee, Jaewoo;Kim, In Kee;Wang, Wei
• 通讯作者: Wang, Wei