CAREER: Advancing Constrained and Non-Convex Learning

职业：推进约束和非凸学习

• 批准号: 2246753
• 负责人: Tianbao Yang
• 金额: $ 52.91万
• 依托单位: Texas A&M Engineering Experiment Station
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-10-01 至 2024-10-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2246753&HistoricalAwards=false
• 关键词: CAREER; Advancing; Constrained; Non; Convex

Machine learning has emerged to be an indispensable tool for addressing many decision-making problems, e.g., autonomous driving. As applications of machine learning algorithms for decision-making broaden and diversify, the requirements on security, fairness, interpretability and generalization have been pushed to higher standards. These emerging issues have brought great challenges to the design of machine learning algorithms in the presence of big and complex data. Traditional machine learning methods by minimizing an unconstrained or simply constrained convex objective have become increasingly unsatisfactory. This project seeks to advance learning with complex objectives and constraints by designing and analyzing efficient and effective optimization algorithms for addressing computational challenges in new machine learning paradigms. The project will enhance the ability to solve large-scale, real-world problems from more diverse and broad applications. Furthermore, the project will strive to communicate the significance of machine learning and optimization and provide excellent research experience to students at different levels.Although both constrained optimization and non-convex optimization have been studied and applied to machine learning in the literature, great challenges and many problems remain unaddressed. The primary focus of this project is to design and analyze a set of efficient optimization algorithms and statistical learning methods for advancing machine learning with complex objectives and constraints at large scale. The technical aims of the project are divided into two thrusts. The first thrust is to (i) develop faster and provable stochastic algorithms for learning with complicated non-convex objectives, and (ii) improve the generalization performance of deep learning by advanced regularization and compression methods through design of efficient optimization algorithms. The second thrust is to (i) design computationally efficient constrained optimization algorithms for learning with complicated and complex constraints, and (ii) investigate their applications in adversarial learning, fair learning, interpretable learning, etc. The optimization tools and techniques developed will enable more advanced regularization and loss minimization methods in machine learning, and should greatly influence other areas, such as operations research, signal processing, data mining, etc.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

机器学习已成为解决许多决策问题的必不可少的工具，例如自动驾驶。随着机器学习算法在决策中的应用扩大和多样化，对安全性，公平性，解释性和概括的要求已被推向更高的标准。这些新兴问题在存在大型和复杂数据的情况下为机器学习算法设计带来了巨大的挑战。 传统的机器学习方法通​​过最大程度地减少不受约束或简单的约束凸目标的方法变得越来越令人满意。该项目旨在通过设计和分析有效有效的优化算法来解决新机器学习范式中的计算挑战，从而通过复杂的目标和约束来提高学习。该项目将增强从更多样化和广泛应用中解决大规模，现实世界中的问题的能力。此外，该项目将努力传达机器学习和优化的重要性，并为不同级别的学生提供出色的研究经验。尽管已研究了受限的优化和非convex优化，并将其应用于文献中的机器学习，巨大的挑战，许多问题仍然尚未得到解决。该项目的主要重点是设计和分析一系列有效的优化算法和统计学习方法，以大规模地使用复杂的目标和约束来推进机器学习。该项目的技术目标分为两个推力。第一个推力是（i）通过复杂的非凸目标来开发用于学习的速度更快，可证明的随机算法，用于学习，（ii）通过设计有效的优化算法，通过高级正则化和压缩方法来改善深度学习的概括性能。第二个力量是（i）设计计算有效的约束优化算法，以进行复杂且复杂的约束，以及（ii）调查它们在对抗性学习，公平学习，可解释的学习等方面的应用。开发的优化工具和技术将启用更高级的正则化和损失方法。法定任务，并被认为是值得通过基金会的智力优点和更广泛影响的审查标准来评估的值得支持的。

项目成果

Multi-block-Single-probe Variance Reduced Estimator for Coupled Compositional Optimization

• DOI: 10.48550/arxiv.2207.08540
• 发表时间: 2022-07
• 期刊: ArXiv
• 作者: Wei Jiang;Gang Li;Yibo Wang-;Lijun Zhang;Tianbao Yang

Large-scale Optimization of Partial AUC in a Range of False Positive Rates

• DOI: 10.48550/arxiv.2203.01505
• 发表时间: 2022-03
• 期刊: ArXiv
• 作者: Yao Yao-Yao;Qihang Lin;Tianbao Yang

Memory-Based Optimization Methods for Model-Agnostic Meta-Learning and Personalized Federated Learning

• 发表时间: 2021-06
• 期刊: J. Mach. Learn. Res.
• 作者: Bokun Wang;Zhuoning Yuan;Yiming Ying;Tianbao Yang

Multi-block Min-max Bilevel Optimization with Applications in Multi-task Deep AUC Maximization

• DOI: 10.48550/arxiv.2206.00260
• 发表时间: 2022-06
• 期刊: ArXiv
• 作者: Quanqi Hu;Yongjian Zhong;Tianbao Yang

Stochastic Methods for AUC Optimization subject to AUC-based Fairness Constraints

• DOI: 10.48550/arxiv.2212.12603
• 发表时间: 2022-12
• 作者: Yao Yao-Yao;Qihang Lin;Tianbao Yang