CAREER: Learning Optimization Algorithms from Data: Interpretability, Reliability, and Scalability

职业：从数据中学习优化算法：可解释性、可靠性和可扩展性

• 批准号: 2145346
• 负责人: Zhangyang Wang
• 金额: $ 50万
• 依托单位: University of Texas at Austin
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-09-15 至 2027-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2145346&HistoricalAwards=false
• 关键词: CAREER; Learning; Optimization; Algorithms; Data

Efficient and scalable optimization algorithms (a.k.a., optimizers) are the cornerstone of almost all computational fields. In many practical applications of optimization, one will repeatedly perform a certain type of optimization tasks over a specific distribution of data. Learning to optimize (L2O) is an emerging paradigm that automatically develops an optimization method (optimizer) by learning from its performance on a set of past optimization tasks. Then on solving new but similar optimization tasks, the learned optimizer can demonstrate many promising benefits including faster convergence and/or better solution quality. As a fast-growing new field, many open challenges remain concerning both L2O's theoretical underpinnings and its practical applicability. In particular, the learned optimizers are often hard to interpret, trust, and scale.The project targets those research gaps and expands to mid-term and long-term research directions pertaining to the foundations of L2O. Specifically, the project proposes a multi-pronged research agenda including: a novel symbolic representation that makes L2O lightweight and more interpretable; a Bayesian L2O modeling framework that can quantify optimizer uncertainty; new customized designs of L2O model architectures and regularizers that can robustly encode problem-specific priors; and a generic amalgamation scheme to bridge L2O training to classical optimizers as teachers. Each thrust addresses a unique aspect of L2O (representation, calibration, model design, and training strategy). Meanwhile, those thrusts are compatible with each other and can be applied together. The proposed efforts synergize cutting-edge technical advances from deep learning, symbolic learning, Bayesian optimization, and meta learning. Successful outcomes are expected to turn L2O into principled science as well as a mature tool for real applications. This project has an integrated plan of result dissemination, education, and outreach. In particular, all new algorithms resulting from the project will be integrated into the Open-L2O software package, developed and maintained by the PI's group.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

高效且可扩展的优化算法（又称优化器）是几乎所有计算领域的基石。在优化的许多实际应用中，人们将对特定的数据分布重复执行某种类型的优化任务。学习优化（L2O）是一种新兴范例，它通过学习过去一组优化任务的性能来自动开发优化方法（优化器）。然后，在解决新的但类似的优化任务时，学习的优化器可以展示许多有希望的好处，包括更快的收敛和/或更好的解决方案质量。作为一个快速发展的新领域，L2O 的理论基础和实际适用性仍然存在许多开放的挑战。特别是，学习到的优化器通常难以解释、信任和扩展。该项目针对这些研究空白，并将其扩展到与 L2O 基础相关的中长期研究方向。具体来说，该项目提出了一个多管齐下的研究议程，包括：一种新颖的符号表示，使 L2O 轻量级且更易于解释；贝叶斯 L2O 建模框架，可以量化优化器的不确定性； L2O 模型架构和正则化器的新定制设计，可以对特定问题的先验进行稳健编码；以及一个通用的合并方案，将 L2O 培训与作为教师的经典优化器联系起来。每个主旨都涉及 L2O 的一个独特方面（表示、校准、模型设计和训练策略）。同时，这些推力是相互兼容、可以共同应用的。所提出的努力协同了深度学习、符号学习、贝叶斯优化和元学习的尖端技术进步。成功的成果有望将 L2O 转变为原理科学以及实际应用的成熟工具。该项目有一个成果传播、教育和推广的综合计划。特别是，该项目产生的所有新算法都将集成到由 PI 小组开发和维护的 Open-L2O 软件包中。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力优势和评估进行评估，被认为值得支持。更广泛的影响审查标准。

项目成果

M-L2O: Towards Generalizable Learning-to-Optimize by Test-Time Fast Self-Adaptation

M-L2O：通过测试时间快速自适应实现可推广的学习优化

• DOI: 10.48550/arxiv.2303.00039
• 发表时间: 2023-02-28
• 期刊: ArXiv
• 作者: Junjie Yang;Xuxi Chen;Tianlong Chen;Zhangyang Wang;Yitao Liang
• 通讯作者: Yitao Liang