Collaborative Research: Consensus and Distributed Optimization in Non-Convex Environments with Applications to Networked Machine Learning

协作研究：非凸环境中的共识和分布式优化及其在网络机器学习中的应用

• 批准号: 2240788
• 负责人: Shahin Shahrampour
• 金额: $ 25万
• 依托单位: Northeastern University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-08-01 至 2026-07-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2240788&HistoricalAwards=false
• 关键词: Collaborative; Research; Consensus; Distributed; Optimization

Distributed optimization is a vehicle for machine learning and data analysis over networks as it provides the means to train models in a scalable fashion. This project aims to advance the fundamental knowledge on distributed non-convex optimization by studying two important classes of non-convex problems. The intellectual merits of the project include investigating distributed retraction-free manifold optimization, and distributed non-smooth weakly-convex optimization. Scientific contributions of this project will bring transformative change to the understanding of consensus and coordination in non-convex environments. The broader impacts of the project include educational components to introduce distributed optimization as a practical tool for the next generation of engineers. These educational plans include seminar presentations for a broader audience as well as advanced course development to introduce state-of-the-art decentralized optimization techniques.The goal of this project is to address distributed approaches for non-convex optimization implemented in heterogeneous computing environments. Retraction-free methods for distributed implementation of Riemannian Stochastic Gradient Descent will be investigated, followed by illustration of the benefits of this technique in reduced rank and sparse regression as well as training of sparse deep neural networks. In both cases, sparsity will be ensured by constraining the search for the model parameters to a lower-dimensional Riemannian manifold. The study will further consider distributed non-smooth, non-convex (weakly-convex) optimization, and develop asynchronous sub-gradient descent methods for communication-efficient optimization. The goal will be to establish both global and local convergence results for the proposed methods.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

分布式优化是通过网络进行机器学习和数据分析的工具，因为它提供了以可扩展的方式训练模型的方法。该项目旨在通过研究两类重要的非凸问题来推进分布式非凸优化的基础知识。该项目的智力优点包括研究分布式无回缩流形优化和分布式非光滑弱凸优化。该项目的科学贡献将为非凸环境中共识和协调的理解带来革命性的变化。该项目更广泛的影响包括教育部分，旨在将分布式优化作为下一代工程师的实用工具引入。这些教育计划包括面向更广泛受众的研讨会演示以及高级课程开发，以介绍最先进的分散优化技术。该项目的目标是解决在异构计算环境中实现非凸优化的分布式方法。将研究分布式实现黎曼随机梯度下降的无回缩方法，然后说明该技术在降阶和稀疏回归以及稀疏深度神经网络训练方面的优势。在这两种情况下，都可以通过将对模型参数的搜索限制在低维黎曼流形来确保稀疏性。该研究将进一步考虑分布式非光滑、非凸（弱凸）优化，并开发用于通信高效优化的异步次梯度下降方法。目标是为所提出的方法建立全球和本地的趋同结果。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。