AF: Small: Advances in Private Optimization

AF：小：私人优化的进展

基本信息

批准号：
2211718
负责人：
Ashok Cutkosky
金额：
$ 60万
依托单位：
Trustees of Boston University
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2023
资助国家：
美国
起止时间：
2023-02-01 至 2026-01-31
项目状态：
未结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2211718&HistoricalAwards=false
关键词：
AF Small Advances Private Optimization

项目摘要

Modern Artificial intelligence (AI) systems are typically built with the use of large datasets that may be generated by individuals contributing data over the internet, such as through product ratings, comments, or other online interactions. As a result, it is critical that such AI systems are able to preserve the privacy of the individuals whose data is used: it should not be possible for an outside observer to learn anything about any individual person through the use of the AI system. This project will investigate the fundamental limits of the trade-off^s between privacy and performance to build AI systems that are as performant as possible without compromising privacy. The project's results will not only improve the security of people who already benefit from products that learn from their data but will also reduce bias in AI by enabling more sensitive or vulnerable individuals to participate safely.On a technical level, this project will develop new differentially private stochastic optimization algorithms. In recent years, there has been a surge of interest in private optimization, but the theory for private non-convex optimization (which is required for training neural networks) is surprisingly underdeveloped. For this setting, the typical approach is to treat a standard non-private algorithm as a black box, providing it with inputs that have already been processed by adding noise to obscure individual contributions so that the output must also necessarily preserve privacy. This project will open up this black box to produce new algorithms that improve privacy guarantees while maintaining good convergence results both in theory and in practice. One of the initial technical approaches will be to develop a connection between momentum, a ubiquitous technique in modern optimization algorithms, with privacy. This approach does not require structural assumptions onthe loss surface (e.g., convexity, smoothness) to ensure privacy but has the potential to significantly decrease the amount of noise injected into the algorithm, resulting in improved performance with the same level of privacy.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

现代人工智能 (AI) 系统通常是使用大型数据集构建的，这些数据集可能是由个人在互联网上提供数据（例如通过产品评级、评论或其他在线交互）生成的。因此，此类人工智能系统能够保护数据使用者的隐私至关重要：外部观察者不可能通过使用人工智能系统了解任何个人的任何信息。该项目将研究隐私和性能之间权衡的基本限制，以构建在不损害隐私的情况下尽可能高性能的人工智能系统。该项目的结果不仅将提高那些已经从从数据中学习的产品中受益的人们的安全性，而且还将通过使更敏感或脆弱的个人能够安全地参与来减少人工智能的偏见。在技术层面上，该项目将开发新的差异化产品私有随机优化算法。近年来，人们对私有优化的兴趣激增，但私有非凸优化理论（训练神经网络所需的）却令人惊讶地不发达。对于这种设置，典型的方法是将标准非私有算法视为黑匣子，为其提供已经通过添加噪声来掩盖个人贡献进行处理的输入，以便输出也必须保护隐私。该项目将打开这个黑匣子，产生新的算法，提高隐私保证，同时在理论和实践中保持良好的收敛结果。最初的技术方法之一是在动量（现代优化算法中普遍存在的技术）与隐私之间建立联系。这种方法不需要对损失表面进行结构假设（例如，凸度、平滑度）来确保隐私，但有可能显着减少注入算法的噪声量，从而在相同隐私级别下提高性能。该奖项反映了通过使用基金会的智力价值和更广泛的影响审查标准进行评估，NSF 的法定使命被认为值得支持。

项目成果

期刊论文数量（7）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

Parameter-free Regret in High Probability with Heavy Tails

DOI：
10.48550/arxiv.2210.14355
发表时间：
2022-10
期刊：
ArXiv
影响因子：
0
作者：
Jiujia Zhang;Ashok Cutkosky
通讯作者：
Jiujia Zhang;Ashok Cutkosky

Optimal Stochastic Non-smooth Non-convex Optimization through Online-to-Non-convex Conversion

DOI：
10.48550/arxiv.2302.03775
发表时间：
2023-02
期刊：
ArXiv
影响因子：
0
作者：
Ashok Cutkosky;Harsh Mehta;Francesco Orabona
通讯作者：
Ashok Cutkosky;Harsh Mehta;Francesco Orabona

Differentially Private Online-to-batch for Smooth Losses

在线批量差异化隐私以实现平滑损失

DOI：
发表时间：
2022
期刊：
Advances in neural information processing systems
影响因子：
0
作者：
Zhang, Qinzi;Tran, Hoang;Cutkosky, Ashok
通讯作者：
Cutkosky, Ashok

Unconstrained Online Learning with Unbounded Losses

DOI：
10.48550/arxiv.2306.04923
发表时间：
2023-06
期刊：
ArXiv
影响因子：
0
作者：
Andrew Jacobsen;Ashok Cutkosky
通讯作者：
Andrew Jacobsen;Ashok Cutkosky

Differentially Private Image Classification from Features

DOI：
10.48550/arxiv.2211.13403
发表时间：
2022-11
期刊：
ArXiv
影响因子：
0
作者：
Harsh Mehta;Walid Krichene;Abhradeep Thakurta;Alexey Kurakin;Ashok Cutkosky
通讯作者：
Harsh Mehta;Walid Krichene;Abhradeep Thakurta;Alexey Kurakin;Ashok Cutkosky