Collaborative Research: CCSS: Learning to Optimize: From New Algorithms to New Theory

合作研究：CCSS：学习优化：从新算法到新理论

• 批准号: 2113904
• 负责人: Zhangyang Wang
• 金额: $ 22万
• 依托单位: University of Texas at Austin
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-08-15 至 2024-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2113904&HistoricalAwards=false
• 关键词: Collaborative; Research; CCSS; Learning; Optimize

Solving machine learning (ML) problems requires efficient and scalable optimization algorithms. State-of-the-art general purpose algorithms often need to compute a large number of iterations and hence have limited applicability to real-time applications. To circumvent this shortcoming, learning to optimize (L2O) methods aim to learn a shorter (i.e., faster) optimization path over a task distribution at meta-training, based on the tasks’ common structures and a more global view of their geometries, and then apply the learned optimizer to new similar tasks at meta-testing. Despite extensive empirical success, the existing L2O methods perform well mainly on optimization tasks with similar structures, but likely perform poorly on out-of-distribution tasks. Furthermore, there has been little theory understanding the convergence and generalization of L2O algorithms. Thus, the proposed program will design novel L2O approaches, so that the trained optimizer can generalize to a broad range of practical tasks, particularly out-of-distribution tasks, and will have guaranteed convergence and generalization performance in L2O training and testing.Specifically, the proposed program will design new L2O approaches with both generalizability to out-of-distribution tasks and safeguarded feature for guaranteed worst-case convergence, will develop a theoretical framework for analyzing the convergence rate for L2O meta-training, and will provide comprehensive characterization of the generalization performance for L2O meta-testing. The new algorithms and theory will be evaluated over applications of on-device model adaptation in internet-of-things (IoT) systems, sparse recovery for images and wireless signals, and algorithmic adaptation in reconfiguration of communication systems. The project is anticipated to significantly mature the field of L2O, and provide training opportunities for a diverse group of students at the new intersection of optimization, machine learning, signal processing, and data science.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

解决机器学习（ML）问题需要有效且可扩展的优化算法。最新的通用算法通常需要计算大量迭代，因此对实时应用程序的可应用性有限。为了避免这一缺点，学习优化（L2O）方法旨在根据任务的共同结构和对几何形状的更全局的看法，学习较短的（即更快）优化路径，然后将学习的优化器应用于Meta-testing的新型任务。尽管经验取得了广泛的成功，但现有的L2O方法主要在具有相似结构的优化任务上表现良好，但在分发任务上可能表现较差。此外，几乎没有理论理解L2O算法的融合和概括。拟议的计划将设计新颖的L2O方法，以便训练有素的优化器可以推广到各种各样的实际任务，尤其是分布式任务，并保证在L2O培训和测试中保证收敛性和概括性性能。具体而言，提议的计划将设计新的L2O方法，以确保范围内的AREDE SAFERISE AR DEARSTARE AR DEATERID SAFERED SAFERED SAFEREDS TEATER SAFERTIBERS TEATER SAFERD SEAFERDS TEATERS TEATERS WEATS TEATERING SERVARTS WESTIST SERVANTS WESSITITS造就，使得范围内的特征，分析L2O元训练的收敛速率以及新算法和理论的理论框架将在启用（IoT）系统中的设备模型适应（IoT）系统的应用中进行评估，图像和无线信号的稀疏恢复以及算法适应通信系统的算法适应。预计该项目将在L2O领域中显着成熟，并在新的优化，机器学习，信号处理和数据科学的交集中为潜水员组提供培训机会。该奖项反映了NSF的法定任务，并被认为是通过基金会的知识分子和更广泛的影响审查标准来通过评估来通过评估来获得的支持。

项目成果

Bayesian Modeling and Uncertainty Quantification for Learning to Optimize: What, Why, and How

• 发表时间: 2022
• 作者: Yuning You;Yue Cao;Tianlong Chen;Zhangyang Wang;Yang Shen

Learning to Optimize Differentiable Games

学习优化可微分游戏

• 发表时间: 2023
• 期刊: International Conference on Machine Learning
• 作者: Chen, Xuxi;Vadori, Nelson;Chen, Tianlong;Wang, Zhangyang
• 通讯作者: Wang, Zhangyang

Learning to Optimize: A Primer and A Benchmark

• 发表时间: 2021-03
• 期刊: J. Mach. Learn. Res.
• 作者: Tianlong Chen;Xiaohan Chen;Wuyang Chen;Howard Heaton;Jialin Liu;Zhangyang Wang;W. Yin

Learning to generalize provably in learning to optimize

在学习优化中学习可证明泛化

• 发表时间: 2023
• 期刊: International Conference on Artificial Intelligence and Statistics (AISTATS
• 作者: Yang, Junjie;Chen, Tianlong;Zhu, Mingkang;He, Fengxiang;Tao, Dacheng;Liang, Yingbin;Wang, Zhangyang.
• 通讯作者: Wang, Zhangyang.