Collaborative Research: CIF: Small: A Unified Framework of Distributional Optimization via Variational Transport

合作研究：CIF：小型：通过变分传输的分布式优化的统一框架

• 批准号: 2008827
• 负责人: Zhaoran Wang
• 金额: $ 24.99万
• 依托单位: Northwestern University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2020
• 资助国家: 美国
• 起止时间: 2020-10-01 至 2023-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2008827&HistoricalAwards=false
• 关键词: Collaborative; Research; CIF; Small; Unified

Distributional optimization refers to a class of mathematical problems where the optimizing variable in the objective function is a probability measure over some space. Because of its highly technical nature, distributional optimization has remained largely unexplored with advances made only on specific problem instances. This project proposes a unified framework to explore challenging distributional optimization problems in a wide range of important application domains. The main objective is to develop a comprehensive theory supporting the principled design of novel and efficient optimization algorithms. To do so, establishing connections between several mathematical disciplines will be required, including optimization theory, optimal transport, functional inequalities, and statistics. This will promote the cross-fertilization of ideas and lead to the creation of training material from an interdisciplinary perspective. The resulting open-source packages will be made available to support research efforts in related fields that our daily lives depend on. Problems in distributional optimization are infinite-dimensional optimization problems where the optimization variable is a probability measure. Many research problems fall into this class of problems; in particular, any non-convex optimization problem over Euclidean space can be cast as a convex distributional optimization problem. Traditionally, specific instances of these problems have been studied independently of each other. Formulating these seemingly different optimization problems into a single unified framework will allow more powerful mathematical techniques and tools to be used. This will lead to deeper insights into the structure of solutions, and to efficient algorithms tailored to large-scale applications in artificial intelligence and data science. The proposed framework is based on optimal transport theory that endows the space of distributions with a natural geometry. The proposed algorithm utilizes the gradient flow of the objective with respect to this geometry. To achieve scalability, the optimization variable is approximated by a collection of particles, with the algorithm now describing the collective dynamics of the particles. A novel variational approach will be used to approximate the gradient descent direction. The theoretical properties of this algorithm will be investigated thoroughly, including provable performance guarantees, convergence rates, and statistical properties. Case studies will be carried out by specializing this unified framework to applications such as Bayesian inference and distributionally robust learning. The acceleration of this algorithm will also be investigated by incorporating existing optimization techniques, such as momentum and variance reduction, as a way to improve convergence rates. Finally, the project will explore how to adapt this algorithm from minimization problems to min-max problems in order to deal with game-theoretic applications.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

分布优化是指一类数学问题，其中目标函数中的优化变量是某个空间上的概率度量。由于其技术性很强，分布式优化在很大程度上尚未得到探索，仅在特定问题实例上取得了进展。该项目提出了一个统一的框架来探索广泛的重要应用领域中具有挑战性的分布式优化问题。主要目标是开发支持新颖高效优化算法原理设计的综合理论。为此，需要在几个数学学科之间建立联系，包括优化理论、最优传输、函数不等式和统计学。这将促进思想的交叉融合，并从跨学科的角度创建培训材料。由此产生的开源包将用于支持我们日常生活所依赖的相关领域的研究工作。分布优化中的问题是无限维优化问题，其中优化变量是概率测度。许多研究问题都属于此类问题；特别是，欧几里德空间上的任何非凸优化问题都可以转化为凸分布优化问题。传统上，这些问题的具体实例是相互独立研究的。将这些看似不同的优化问题表述为一个统一的框架将允许使用更强大的数学技术和工具。这将导致对解决方案结构的更深入了解，以及针对人工智能和数据科学大规模应用量身定制的高效算法。所提出的框架基于最优传输理论，赋予分布空间自然的几何形状。所提出的算法利用了目标相对于该几何形状的梯度流。为了实现可扩展性，优化变量由粒子集合来近似，算法现在描述粒子的集体动态。将使用一种新颖的变分方法来近似梯度下降方向。该算法的理论特性将得到彻底研究，包括可证明的性能保证、收敛速度和统计特性。案例研究将通过将这个统一框架专门应用于贝叶斯推理和分布式鲁棒学习等应用来进行。还将通过结合现有的优化技术（例如动量和方差减少）来研究该算法的加速，作为提高收敛速度的一种方法。最后，该项目将探索如何将该算法从最小化问题调整为最小-最大问题，以处理博弈论应用。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力优势和评估进行评估，认为值得支持。更广泛的影响审查标准。