AF: Small: Advances in Private Optimization

AF：小：私人优化的进展

基本信息

批准号：
2211718
负责人：
Ashok Cutkosky
金额：
$ 60万
依托单位：
Trustees of Boston University
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2023
资助国家：
美国
起止时间：
2023-02-01 至 2026-01-31
项目状态：
未结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=2211718&HistoricalAwards=false
关键词：
AF Small Advances Private Optimization

项目摘要

Modern Artificial intelligence (AI) systems are typically built with the use of large datasets that may be generated by individuals contributing data over the internet, such as through product ratings, comments, or other online interactions. As a result, it is critical that such AI systems are able to preserve the privacy of the individuals whose data is used: it should not be possible for an outside observer to learn anything about any individual person through the use of the AI system. This project will investigate the fundamental limits of the trade-off^s between privacy and performance to build AI systems that are as performant as possible without compromising privacy. The project's results will not only improve the security of people who already benefit from products that learn from their data but will also reduce bias in AI by enabling more sensitive or vulnerable individuals to participate safely.On a technical level, this project will develop new differentially private stochastic optimization algorithms. In recent years, there has been a surge of interest in private optimization, but the theory for private non-convex optimization (which is required for training neural networks) is surprisingly underdeveloped. For this setting, the typical approach is to treat a standard non-private algorithm as a black box, providing it with inputs that have already been processed by adding noise to obscure individual contributions so that the output must also necessarily preserve privacy. This project will open up this black box to produce new algorithms that improve privacy guarantees while maintaining good convergence results both in theory and in practice. One of the initial technical approaches will be to develop a connection between momentum, a ubiquitous technique in modern optimization algorithms, with privacy. This approach does not require structural assumptions onthe loss surface (e.g., convexity, smoothness) to ensure privacy but has the potential to significantly decrease the amount of noise injected into the algorithm, resulting in improved performance with the same level of privacy.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

现代人工智能（AI）系统通常是通过使用大型数据集构建的，这些数据集可能是由个人通过互联网贡献数据（例如通过产品评分，评论或其他在线互动）生成的。结果，至关重要的是，这样的AI系统能够保留使用数据的个人的隐私：外部观察者不应该通过使用AI系统来了解任何个人。该项目将调查隐私与绩效之间权衡的基本限制，以建立在不损害隐私的情况下尽可能绩效的AI系统。该项目的结果不仅将提高已经从数据中学习的产品中受益的人的安全性，而且还将通过使更敏感或更脆弱的个人安全地参与AI来减少AI的偏见。在技术层面上，该项目将开发新的私人私人随机优化算法。近年来，人们对私人优化产生了兴趣，但是私人非凸优化的理论（训练神经网络所必需）的理论令人惊讶地欠发达。对于这种设置，典型的方法是将标准的非私有算法视为黑匣子，为其提供已经通过添加噪声来模糊个人贡献来处理的输入，以便输出还必须保留隐私。该项目将打开这个黑匣子，以产生新的算法，以改善隐私保证，同时在理论和实践中保持良好的收敛结果。最初的技术方法之一是在现代优化算法中具有隐私的动量（一种无处不在的技术）之间建立联系。这种方法不需要在损失表面（例如，凸度，光滑度）上进行结构性假设，以确保隐私，但有可能显着减少注入算法中的噪声量，从而提高了具有相同隐私水平的绩效。该奖项颁发了NSF的法定任务，并通过评估范围的范围来表现出众所周知的支持。

项目成果

期刊论文数量（7）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

Parameter-free Regret in High Probability with Heavy Tails

DOI：
10.48550/arxiv.2210.14355
发表时间：
2022-10
期刊：
ArXiv
影响因子：
0
作者：
Jiujia Zhang;Ashok Cutkosky
通讯作者：
Jiujia Zhang;Ashok Cutkosky

Optimal Stochastic Non-smooth Non-convex Optimization through Online-to-Non-convex Conversion

DOI：
10.48550/arxiv.2302.03775
发表时间：
2023-02
期刊：
ArXiv
影响因子：
0
作者：
Ashok Cutkosky;Harsh Mehta;Francesco Orabona
通讯作者：
Ashok Cutkosky;Harsh Mehta;Francesco Orabona

Differentially Private Online-to-batch for Smooth Losses

在线批量差异化隐私以实现平滑损失

DOI：
发表时间：
2022
期刊：
Advances in neural information processing systems
影响因子：
0
作者：
Zhang, Qinzi;Tran, Hoang;Cutkosky, Ashok
通讯作者：
Cutkosky, Ashok

Unconstrained Online Learning with Unbounded Losses

DOI：
10.48550/arxiv.2306.04923
发表时间：
2023-06
期刊：
ArXiv
影响因子：
0
作者：
Andrew Jacobsen;Ashok Cutkosky
通讯作者：
Andrew Jacobsen;Ashok Cutkosky

Differentially Private Image Classification from Features

DOI：
10.48550/arxiv.2211.13403
发表时间：
2022-11
期刊：
ArXiv
影响因子：
0
作者：
Harsh Mehta;Walid Krichene;Abhradeep Thakurta;Alexey Kurakin;Ashok Cutkosky
通讯作者：
Harsh Mehta;Walid Krichene;Abhradeep Thakurta;Alexey Kurakin;Ashok Cutkosky