CAREER: Structured Minimax Optimization: Theory, Algorithms, and Applications in Robust Learning

职业：结构化极小极大优化：稳健学习中的理论、算法和应用

• 批准号: 2338846
• 负责人: Aryan Mokhtari
• 金额: $ 66万
• 依托单位: University of Texas at Austin
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2024
• 资助国家: 美国
• 起止时间: 2024-08-01 至 2029-07-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2338846&HistoricalAwards=false
• 关键词: CAREER; Structured; Minimax; Optimization; Theory; CAREER; Structured; Minimax; Optimization; Theory

This proposal seeks to extend the mathematical theory of minimax optimization, a very commonly used approach that has played a pivotal role in advancing the fields of information theory, machine learning and signal processing. Notably, there has been a recent surge of interest in minimax optimization due to its critical relevance in artificial intelligence (AI), where it can be used to make deep neural networks more resilient against adversarial disturbances in the underlying distribution of data. While there has been recent progress in enhancing the theory of minimax optimization and its algorithms, a notable gap persists in the applicability of this theory to real-world AI scenarios. Existing algorithms and theory primarily focus on the so-called convex-concave optimization setting, while contemporary learning applications frequently entail minimax problems that do not adhere to this structure and are more complex. This project aims to develop efficient methods for minimax optimization for AI by capitalizing on the unique structure of the AI prediction function. This interdisciplinary project integrates research findings into graduate and undergraduate courses and promotes STEM interest among high school students. The overarching goal of this project is to advance optimization theory and algorithms for robust machine learning model training by investigating their structured minimax optimization problems. The minimization problems in robust learning are structured based on the prediction function model, and we will investigate different types of models, such as neural networks. For each considered prediction function, the minimization component of the minimax problem is nonconvex; nonetheless, they each possess a distinct structure that we meticulously explore in separate thrusts. Moreover, within each thrust, we study different types of structured inner maximization problems motivated by the three applications closely examined in this project: distributionally robust learning, adversarially robust training, and discrete robust learning. The developed algorithms can significantly advance these application areas in terms of computational cost and accuracy.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

该提案旨在扩展最小值优化的数学理论，这是一种非常常用的方法，在推进信息理论，机器学习和信号处理领域方面发挥了关键作用。值得注意的是，由于它在人工智能（AI）中的关键相关性，最近人们对最小值优化的兴趣激增，在该优化中，它可以用来使深层神经网络在数据的潜在分布中更具弹性。尽管最近在增强了最小值优化及其算法的理论方面取得了进展，但该理论对现实世界中的AI场景的适用性持续了一个显着的差距。现有的算法和理论主要集中于所谓的凸孔凸优化设置，而当代学习应用程序经常需要不遵守这种结构并且更复杂的minimax问题。该项目旨在通过利用AI预测函数的独特结构来开发为AI最小化优化的有效方法。这个跨学科项目将研究结果纳入研究生和本科课程，并促进了高中生的STEM兴趣。该项目的总体目标是通过研究其结构化的最小值优化问题来推进强大机器学习模型培训的优化理论和算法。鲁棒学习中的最小化问题是根据预测函数模型构造的，我们将研究不同类型的模型，例如神经网络。对于每个考虑的预测函数，最小值问题的最小化组成部分是非convex。尽管如此，它们每个人都有一个独特的结构，我们在不同的推力中精心探索。此外，在每个推力中，我们研究了该项目中仔细检查的三个应用程序所激发的不同类型的结构化内部最大化问题：分布稳健的学习，对抗性稳健的训练和离散的鲁棒学习。开发的算法可以从计算成本和准确性方面显着提高这些应用领域。该奖项反映了NSF的法定任务，并且使用基金会的知识分子优点和更广泛的影响审查标准，被认为值得通过评估来获得支持。