CAREER: Learning Optimization Algorithms from Data: Interpretability, Reliability, and Scalability

职业：从数据中学习优化算法：可解释性、可靠性和可扩展性

• 批准号: 2145346
• 负责人: Zhangyang Wang
• 金额: $ 50万
• 依托单位: University of Texas at Austin
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-09-15 至 2027-08-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2145346&HistoricalAwards=false
• 关键词: CAREER; Learning; Optimization; Algorithms; Data

Efficient and scalable optimization algorithms (a.k.a., optimizers) are the cornerstone of almost all computational fields. In many practical applications of optimization, one will repeatedly perform a certain type of optimization tasks over a specific distribution of data. Learning to optimize (L2O) is an emerging paradigm that automatically develops an optimization method (optimizer) by learning from its performance on a set of past optimization tasks. Then on solving new but similar optimization tasks, the learned optimizer can demonstrate many promising benefits including faster convergence and/or better solution quality. As a fast-growing new field, many open challenges remain concerning both L2O's theoretical underpinnings and its practical applicability. In particular, the learned optimizers are often hard to interpret, trust, and scale.The project targets those research gaps and expands to mid-term and long-term research directions pertaining to the foundations of L2O. Specifically, the project proposes a multi-pronged research agenda including: a novel symbolic representation that makes L2O lightweight and more interpretable; a Bayesian L2O modeling framework that can quantify optimizer uncertainty; new customized designs of L2O model architectures and regularizers that can robustly encode problem-specific priors; and a generic amalgamation scheme to bridge L2O training to classical optimizers as teachers. Each thrust addresses a unique aspect of L2O (representation, calibration, model design, and training strategy). Meanwhile, those thrusts are compatible with each other and can be applied together. The proposed efforts synergize cutting-edge technical advances from deep learning, symbolic learning, Bayesian optimization, and meta learning. Successful outcomes are expected to turn L2O into principled science as well as a mature tool for real applications. This project has an integrated plan of result dissemination, education, and outreach. In particular, all new algorithms resulting from the project will be integrated into the Open-L2O software package, developed and maintained by the PI's group.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

有效且可扩展的优化算法（又称优化器）是几乎所有计算场的基石。在许多实用的优化应用中，人们将在特定的数据分布上反复执行某种类型的优化任务。学习优化（L2O）是一种新兴范式，它通过在过去的一系列优化任务上从性能中学习，从而自动开发优化方法（优化器）。然后，在解决新的但类似的优化任务时，学习的优化器可以证明许多有希望的好处，包括更快的收敛和/或更好的解决方案质量。作为一个快速增长的新领域，关于L2O的理论基础及其实际适用性的许多开放挑战仍然存在。特别是，学到的优化者通常很难解释，信任和规模。该项目针对这些研究差距，并扩展到与L2O基础有关的中期和长期研究方向。具体而言，该项目提出了一个多管齐的研究议程，包括：一种新颖的符号表示，使L2O轻量级和更容易解释；可以量化优化器不确定性的贝叶斯L2O建模框架； L2O模型体系结构和正规化器的新定制设计，这些设计可以鲁棒编码特定问题的先验；以及一种通用的合并计划，将L2O培训桥接到经典优化者作为教师。每个推力都解决了L2O（表示，校准，模型设计和培训策略）的独特方面。同时，这些推力彼此兼容，可以一起应用。拟议的努力从深度学习，象征性学习，贝叶斯优化和元学习中提出了尖端的技术进步。预计成功的结果将把L2O变成原始的科学以及用于实际应用的成熟工具。该项目制定了结果传播，教育和外展的综合计划。特别是，该项目产生的所有新算法都将集成到PI小组开发和维护的Open-L2O软件包中。该奖项反映了NSF的法定任务，并认为值得通过基金会的智力优点和更广泛的影响评估标准通过评估来获得支持。

项目成果

M-L2O: Towards Generalizable Learning-to-Optimize by Test-Time Fast Self-Adaptation

• DOI: 10.48550/arxiv.2303.00039
• 发表时间: 2023-02
• 期刊: ArXiv
• 作者: Junjie Yang;Xuxi Chen;Tianlong Chen;Zhangyang Wang;Yitao Liang