RI: Small: A Study of New Aggregate Losses for Machine Learning

RI：小：机器学习新总损失的研究

• 批准号: 2008532
• 负责人: Siwei Lyu
• 金额: $ 45万
• 依托单位: SUNY at Albany
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-10-01 至 2020-12-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2008532&HistoricalAwards=false
• 关键词: RI; Small; Study; New; Aggregate

Machine learning is instrumental for the recent advances in AI and big data analysis. They have been used in almost every area of computer science and many fields of natural sciences, engineering, and social sciences. The central task of machine learning is to “train” a model, which entails seeking models that minimize certain performance metrics over a set of training examples. Such performance metrics are termed as the aggregate losses, which are to be distinguished from the individual losses that measures the quality of the model on a single training example. As the link between the training data and the model to be learned, the aggregate loss is a fundamental component in machine learning algorithms, and its theoretical and practical significance warrants a comprehensive and systematic study. The proposed work will focus on several fundamental research questions concerning the aggregate loss: are there any other types of aggregate loss beyond the average individual losses?; if so, what will be a general abstract formulation of these new aggregate loss?; how can the new aggregate losses be adapted to different machine learning problems?; and what are the statistical and computational behaviors of machine learning algorithms using the general aggregate losses?. The technical aims of the project are divided into four interrelated thrusts. The first thrust explores new types of rank-based aggregate losses for binary classification and study efficient algorithms optimizing learning objectives formed based upon them. The new aggregate losses will be applied to problems such as object detection, where rank-based evaluation metric is used dominantly. The second thrust aims to deepen our understanding of the binary classification algorithms developed using the rank-based aggregate losses and will be focused on a study of their statistical theories such as generalization and consistency. The third thrust will extend the study of new types of aggregate losses to other supervised problems (multi-class and multi-label learning and supervised metric learning) and unsupervised learning. The fourth thrust dedicates to the theoretical aspects of aggregate losses, in which an aggregate loss will be abstracted as a set function that maps the ensemble of individual losses to a number. This abstraction will be exploited to study the properties of new aggregate losses that make them superior than the average loss and propose new aggregate losses beyond rank-based ones.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

机器学习对AI和大数据分析的最新进展至关重要。它们几乎都用于计算机科学的每个领域以及许多自然科学，工程和社会科学领域。机器学习的核心任务是“训练”模型，该模型需要寻求模型，以最大程度地减少一组培训示例的某些性能指标。这种性能指标被称为总损失，这些损失应与单个训练示例中测量模型质量的个体损失区分开。随着培训数据与要学习的模型之间的联系，总损失是机器学习算法中的基本组成部分，其理论和实际意义值得一项全面而系统的研究。拟议的工作将重点放在有关总损失的几个基本研究问题上：除了平均个人损失之外，还有其他类型的总损失吗？如果是这样，这些新总损失的一般抽象公式是什么？新的总损失如何适应不同的机器学习问题？使用一般总损失的机器学习算法的统计和计算行为是什么？该项目的技术目标分为四个相互关联的推力。第一个推力探讨了基于等级的二进制分类的新型类型的总体损失，并研究有效的算法优化了基于它们形成的学习对象。新的总损失将应用于诸如对象检测等问题，其中主要使用基于等级的评估度量。第二个推力旨在加深我们对使用基于等级的骨料损失开发的二进制分类算法的理解，并将集中于对其统计理论（例如概括性和一致性）的研究。第三个推力将将新类型的总损失类型的研究扩展到其他监督问题（多级和多标签学习以及监督指标学习）和无监督的学习。第四个推力专门针对总损失的理论方面，其中总损失将被抽象为将个体损失的集合映射到一个数字的集合函数。该抽象将被探讨以研究新的总损失的特性，这些损失使它们比平均损失和提案超过基于等级的损失优越。该奖项反映了NSF的法定任务，并被认为是通过基金会的智力优点和更广泛的影响标准通过评估来评估的。

项目成果

Learning by Minimizing the Sum of Ranked Range

• 发表时间: 2020-10
• 期刊: ArXiv
• 作者: Shu Hu;Yiming Ying;Xin Wang;Siwei Lyu

Stability and differential privacy of stochastic gradient descent for pairwise learning with non-smooth loss

非平滑损失成对学习的随机梯度下降的稳定性和差分隐私

• 发表时间: 2021
• 期刊: Proceedings of the 24th International Conference on Artificial Intelligence and Statistics (AISTATS
• 作者: Yang, Zhenhuan;Lei, Yunwen;Lyu, Siwei;Ying, Yiming
• 通讯作者: Ying, Yiming

Differentially Private SGDA for Minimax Problems

• 发表时间: 2022-01
• 期刊: ArXiv
• 作者: Zhenhuan Yang;Shu Hu;Yunwen Lei;Kush R. Varshney;Siwei Lyu;Yiming Ying

Unmixing Biological Fluorescence Image Data with Sparse and Low-Rank Poisson Regression

• DOI: 10.1101/2023.01.06.523044
• 发表时间: 2023-01
• 期刊: bioRxiv
• 作者: Ruogu Wang;A. Lemus;Colin M. Henneberry;Yiming Ying;Yunlong Feng;A. Valm

Stability and Generalization for Markov Chain Stochastic Gradient Methods

• DOI: 10.48550/arxiv.2209.08005
• 发表时间: 2022-09
• 期刊: ArXiv
• 作者: Puyu Wang;Yunwen Lei;Yiming Ying;Ding-Xuan Zhou