Collaborative Research: Consensus and Distributed Optimization in Non-Convex Environments with Applications to Networked Machine Learning

协作研究：非凸环境中的共识和分布式优化及其在网络机器学习中的应用

• 批准号: 2240788
• 负责人: Shahin Shahrampour
• 金额: $ 25万
• 依托单位: Northeastern University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2023
• 资助国家: 美国
• 起止时间: 2023-08-01 至 2026-07-31
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2240788&HistoricalAwards=false
• 关键词: Collaborative; Research; Consensus; Distributed; Optimization

Distributed optimization is a vehicle for machine learning and data analysis over networks as it provides the means to train models in a scalable fashion. This project aims to advance the fundamental knowledge on distributed non-convex optimization by studying two important classes of non-convex problems. The intellectual merits of the project include investigating distributed retraction-free manifold optimization, and distributed non-smooth weakly-convex optimization. Scientific contributions of this project will bring transformative change to the understanding of consensus and coordination in non-convex environments. The broader impacts of the project include educational components to introduce distributed optimization as a practical tool for the next generation of engineers. These educational plans include seminar presentations for a broader audience as well as advanced course development to introduce state-of-the-art decentralized optimization techniques.The goal of this project is to address distributed approaches for non-convex optimization implemented in heterogeneous computing environments. Retraction-free methods for distributed implementation of Riemannian Stochastic Gradient Descent will be investigated, followed by illustration of the benefits of this technique in reduced rank and sparse regression as well as training of sparse deep neural networks. In both cases, sparsity will be ensured by constraining the search for the model parameters to a lower-dimensional Riemannian manifold. The study will further consider distributed non-smooth, non-convex (weakly-convex) optimization, and develop asynchronous sub-gradient descent methods for communication-efficient optimization. The goal will be to establish both global and local convergence results for the proposed methods.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

分布式优化是通过网络进行机器学习和数据分析的工具，因为它提供了以可扩展方式训练模型的方法。该项目旨在通过研究两个重要类别的非凸问题类别来推进分布式非凸优化的基本知识。该项目的智力优点包括研究分布式无缩回的流形优化和分布式非平滑弱征优化的优化。该项目的科学贡献将使对非凸环境中共识和协调的理解带来变革性的变化。该项目的更广泛影响包括教育组件，以引入分布式优化，作为下一代工程师的实用工具。这些教育计划包括针对更广泛的受众群体的研讨会演讲以及高级课程开发，以介绍最先进的分散优化技术。该项目的目的是解决在异质计算环境中实现的非convex优化的分布式方法。将研究用于分布式实施Riemannian随机梯度下降的无缩回方法，然后进行示例，说明了该技术在降低和稀疏回归中的好处，以及对稀疏的深神经网络的培训。在这两种情况下，都将通过将模型参数的搜索限制为较低的riemannian歧管来确保稀疏性。该研究将进一步考虑分布的非平滑，非凸（弱凸）优化，并开发出异步的次级梯度下降方法，以进行沟通优化。目的是为拟议方法建立全球和本地融合结果。该奖项反映了NSF的法定任务，并使用基金会的知识分子优点和更广泛的影响审查标准，被认为值得通过评估来获得支持。