CIF: Small: Towards Robust Statistical Learning: Theory and Algorithms

CIF：小：迈向稳健的统计学习：理论和算法

基本信息

批准号：
1908905
负责人：
Stanislav Minsker
金额：
$ 35.13万
依托单位：
University of Southern California
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2019
资助国家：
美国
起止时间：
2019-10-01 至 2023-09-30
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1908905&HistoricalAwards=false
关键词：
CIF Small Towards Robust Statistical

项目摘要

Machine learning algorithms are used to automate various tasks by finding patterns in the existing data. The mathematical analysis of machine learning algorithms starts by assuming that the available dataset is described by a model with certain properties. However, as real-world data often do not satisfy the model assumptions exactly, there is a need to reduce the gap between the "mathematical" and "real" worlds by weakening the mathematical assumptions. The concept of robustness plays a central role in understanding this gap. First, the project will formulate principles for building robust algorithms. The project will then apply these principles to address problems related to the existence of mathematically justified and computationally efficient robust methods for prediction and classification tasks, which are among the most popular problems solved by machine learning algorithms. The project will also support undergraduate research by training students to apply advanced methods to the analysis of modern data sets. Additional efforts will be made to establish closer ties between the academic and industry machine learning research communities. One part of the project is devoted to robust empirical risk minimization. Empirical risk minimization is one of the fundamental concepts underlying modern mathematical statistics and statistical learning algorithms, including regression and maximum likelihood estimation. However, empirical risk minimization is not robust in many scenarios, with a single "atypical point" amongst the observations possibly significantly affecting performance. The work done in the course of this project will lead to algorithms that avoid explicit outlier detection and removal, and which instead take advantage of existing or purposefully induced symmetries in the distribution of the data. The analysis of these new algorithms will require the development of novel techniques related to Bahadur-type representations of robust estimators, and of new generalizations of the median-of-means principle. Another part of the project aims at developing robust modifications of the Federated Learning algorithm, originally designed as a communication-effective alternative to the standard centralized datacenter framework. The project will design new and robust versions of the Federated Learning algorithm that provably work in the challenging scenario where the input data have different distributions. Finally, the investigator will address inferential problems in robust learning by devising robust versions of posterior distributions that are central objects in Bayesian statistics; he will study the Bernstein-von Mises theorem for these robust posteriors, a fundamental result connecting the frequentist and Bayesian methods.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

机器学习算法用于通过查找现有数据中的模式来自动执行各种任务。机器学习算法的数学分析首先假设可用数据集由具有某些属性的模型描述。然而，由于现实世界的数据往往不能完全满足模型假设，因此需要通过削弱数学假设来缩小“数学”世界和“现实”世界之间的差距。鲁棒性的概念在理解这一差距方面发挥着核心作用。首先，该项目将制定构建稳健算法的原则。然后，该项目将应用这些原理来解决与预测和分类任务的数学合理且计算高效的稳健方法的存在相关的问题，这些问题是机器学习算法解决的最流行的问题之一。该项目还将通过培训学生应用先进方法来分析现代数据集来支持本科生研究。我们将做出更多努力，在学术界和行业机器学习研究社区之间建立更紧密的联系。该项目的一部分致力于稳健的经验风险最小化。经验风险最小化是现代数学统计学和统计学习算法的基本概念之一，包括回归和最大似然估计。然而，经验风险最小化在许多情况下并不稳健，观察结果中的单个“非典型点”可能会显着影响性能。该项目过程中完成的工作将产生避免显式异常值检测和删除的算法，而是利用数据分布中现有的或有意诱导的对称性。对这些新算法的分析将需要开发与鲁棒估计量的巴哈杜尔型表示相关的新技术，以及中位数原理的新概括。该项目的另一部分旨在开发联邦学习算法的稳健修改，该算法最初被设计为标准集中式数据中心框架的有效通信替代方案。该项目将设计新的、强大的联邦学习算法版本，该算法已被证明可以在输入数据具有不同分布的挑战性场景中工作。最后，研究者将通过设计作为贝叶斯统计中核心对象的后验分布的鲁棒版本来解决鲁棒学习中的推理问题；他将研究这些稳健后验的伯恩斯坦-冯·米塞斯定理，这是连接频率论和贝叶斯方法的基本结果。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力优点和更广泛的影响审查标准进行评估，被认为值得支持。