Splitting methods in optimization: Beyond consistency and convexity.

优化中的分割方法：超越一致性和凸性。

• 批准号: RGPIN-2019-04803
• 负责人: Moursi, Walaa
• 金额: $ 2.4万
• 依托单位: University of Waterloo
• 依托单位国家: 加拿大
• 项目类别: Discovery Grants Program - Individual
• 财政年份: 2022
• 资助国家: 加拿大
• 起止时间: 2022-01-01 至 2023-12-31
• 项目状态: 已结题
• 来源: https://www.nserc-crsng.gc.ca/ase-oro/Details-Detailles_eng.asp?id=749574
• 关键词: Splitting; methods; optimization; Beyond; consistency

Optimization problems, which appear in many fields of science, engineering and healthcare, ask to find minimizers or maximizers of functions subject to constraints. Splitting algorithms are powerful optimization methods which aim to solve constrained convex (but not necessarily smooth) optimization problems, which are oftentimes encountered in practice. Examples of these methods are the Douglas-Rachford method, the forward-backward method and the alternating direction method of multipliers (ADMM). Concrete areas of applications include image processing (e.g., image denoising problems), data science (e.g., support vector machine problems), statistics and machine learning (e.g., the least absolute shrinkage and selection operator (LASSO) problem) and finance (e.g., portfolio optimization). Finding conditions that guarantee the convergence of these algorithms to a solution is a very active area of research. Typical convergence results require the convexity of the objective function and/or the constraints, as well as the existence of solutions. The long-term objective of my research program is to investigate the behaviour of splitting methods in the absence of minimizers and/or convexity. The importance of investigating this situation stems from the fact that it occurs in practice but is not well understood. These aspects of my research connect to phase retrieval, intensity-modulated radiation therapy, and deep learning (via the popular but nonconvex loss function from neural networks). The short-term objectives of my research program are summarized below. 1. Using tools from convex and variational analysis, I will explore the analytic behaviour of different splitting algorithms in the inconsistent and/or nonconvex case. Some of these methods have shown great promise when applied to these cases; however, an in-depth analysis is still lacking. 2. Building on these analytical results, I will utilize techniques from dynamical systems and nonlinear operator theory to develop the algorithmic counterpart of the analysis. In this case, we must deal with operators (for instance, projections onto nonconvex sets) that lack properties typically utilized in convergence proofs. 3. The third component of my research program is the computational counterpart. This research includes numerical testing and comparison of the computational cost of these methods to other existing ones, as well as developing software packages. Impact: On the one hand, the progress we make will be of significant interest for scientists employing splitting methods as well as the industrial and healthcare sectors, since the functions used to model industrial problems are often nonconvex. On the other hand, my students and postdoctoral fellows will acquire a strong analytic and computational foundation. This will contribute to developments in optimization theory and related software. Both aspects will significantly contribute to Canada's leading role in science and technology.

优化问题出现在许多科学，工程和医疗保健领域，要求找到受限制的功能的最小化或最大化器。分裂算法是强大的优化方法，旨在解决受约束的凸（但不一定是平滑）优化问题，这在实践中经常遇到。这些方法的示例是Douglas-rachford方法，前向方法和乘数的交替方向方法（ADMM）。应用程序的具体领域包括图像处理（例如，图像denoising问题），数据科学（例如，支持向量机问题），统计和机器学习（例如，绝对绝对的收缩和选择操作员（LASSO）问题）和财务（例如，投资组合优化）。找到确保这些算法融合溶液的条件是一个非常活跃的研究领域。典型的收敛结果需要目标函数和/或约束以及解决方案的存在。我的研究计划的长期目标是在没有最小化器和/或凸度的情况下研究分裂方法的行为。调查这种情况的重要性源于它在实践中发生，但并未得到充分理解。我的研究的这些方面与相位检索，强度调节的放射疗法和深度学习有关（通过神经网络的流行但非凸的损失函数）。下面总结了我的研究计划的短期目标。 1。使用凸和变异分析中的工具，我将探讨在不一致和/或非凸案情况下不同分裂算法的分析行为。这些方法中的某些方法在应用这些情况下显示出巨大的希望。但是，仍然缺乏深入的分析。 2。在这些分析结果的基础上，我将利用动力学系统和非线性操作者理论的技术来开发分析的算法对应物。在这种情况下，我们必须与通常缺乏在收敛证明中使用的属性的运营商（例如，对非convex集的投影）打交道。 3。我的研究计划的第三个组成部分是计算对应物。这项研究包括这些方法与其他现有方法的计算成本以及开发软件包的数值测试和比较。影响：一方面，我们所取得的进步对于采用分裂方法以及工业和医疗保健领域的科学家来说将具有重大兴趣，因为用于建模工业问题的功能通常是非convex的。另一方面，我的学生和博士后研究员将获得强大的分析和计算基础。这将有助于优化理论和相关软件的发展。这两个方面都将为加拿大在科学技术中的领导作用做出重大贡献。