CIF: Small: High-Dimensional Analysis of Stochastic Iterative Algorithms for Signal Estimation

CIF：小：信号估计随机迭代算法的高维分析

• 批准号: 1718698
• 负责人: Yue Lu
• 金额: $ 51.56万
• 依托单位: Harvard University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2017
• 资助国家: 美国
• 起止时间: 2017-07-01 至 2020-06-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1718698&HistoricalAwards=false
• 关键词: CIF; Small; Dimensional; Analysis; Stochastic

Optimization lies at the heart of modern signal and information processing. In recent years, the soaring quantity of information that is being acquired and becoming available makes computational and algorithmic issues increasingly important. This project contributes to an understanding of the fundamental limits of various stochastic optimization algorithms when dealing with high-dimensional data. Since such algorithms are the workhorse in many estimation, inference, and machine learning tasks, this research is well-posed to make significant and broad impact on many applications. Examples include real-time or low-latency medical image reconstructions, distributed computation on power grids, and the training of artificial neural networks for image understanding.In this project, the PI studies a family of efficient stochastic iterative algorithms for solving large-scale convex and nonconvex optimization problems that arise in various signal estimation tasks. The broad goal in this project is to analyze the exact dynamics of these stochastic iterative algorithms in the high-dimensional limit. This asymptotic analysis provides a complete characterization of the typical behavior of the algorithms. The theoretical investigation draws upon techniques from the statistical physics of mean-field interactive particle systems, the weak convergence theory of stochastic processes, signal processing, information theory, and optimization. The theoretical analysis can be used to clarify the effectiveness of such stochastic methods for large-scale optimization and to establish their fundamental performance bounds. The insights obtained from the analysis can also be used to guide the design of new scalable algorithms to achieve optimal trade-offs between estimation accuracy, sample complexity, and computational complexity.

优化是现代信号和信息处理的核心。近年来，获取和可用的信息量不断增加，使得计算和算法问题变得越来越重要。该项目有助于理解处理高维数据时各种随机优化算法的基本限制。由于此类算法是许多估计、推理和机器学习任务的主力，因此这项研究可以对许多应用产生重大而广泛的影响。示例包括实时或低延迟的医学图像重建、电网上的分布式计算以及用于图像理解的人工神经网络的训练。在该项目中，PI 研究了一系列用于求解大规模凸问题的高效随机迭代算法。以及各种信号估计任务中出现的非凸优化问题。该项目的总体目标是分析这些随机迭代算法在高维限制下的精确动态。这种渐近分析提供了算法典型行为的完整表征。理论研究借鉴了平均场交互粒子系统的统计物理学、随机过程的弱收敛理论、信号处理、信息论和优化等技术。理论分析可用于阐明此类随机方法对于大规模优化的有效性，并确定其基本性能范围。从分析中获得的见解还可用于指导新的可扩展算法的设计，以实现估计精度、样本复杂性和计算复杂性之间的最佳权衡。

项目成果

A Modern Perspective on Streaming PCA and Subspace Tracking: The Missing Data Case

流式 PCA 和子空间跟踪的现代视角：丢失数据案例

• 发表时间: 2018-10
• 期刊: Proceedings of the IEEE
• 影响因子: 20.6
• 作者: Chi, Y.;Balzano, L;Lu, Y. M.
• 通讯作者: Lu, Y. M.

Fundamental Limits of PhaseMax for Phase Retrieval: A Replica Analysis

PhaseMax 用于相位检索的基本限制：副本分析

• 发表时间: 2017-10
• 期刊: 7th IEEE Workshop on Computational Advances in Multi-Sensor Adaptive Processing (CAMSAP
• 作者: Dhifallah, O.;Lu, Y. M.
• 通讯作者: Lu, Y. M.

Optimal Spectral Initialization for Signal Recovery With Applications to Phase Retrieval

用于相位检索应用的信号恢复的最佳频谱初始化

• DOI: 10.1109/tsp.2019.2904918
• 发表时间: 2018-11-11
• 期刊: IEEE Transactions on Signal Processing
• 影响因子: 5.4
• 作者: Wangyu Luo;Wael Alghamdi;Yue M. Lu
• 通讯作者: Yue M. Lu

The Scaling Limit of High-Dimensional Online Independent Component Analysis

高维在线独立成分分析的缩放极限

• 发表时间: 2017-10
• 期刊: Conference on Neural Information Processing Systems (NIPS
• 作者: Wang, C;Lu, Y. M.
• 通讯作者: Lu, Y. M.

The role of regularization in classification of high-dimensional noisy Gaussian mixture

正则化在高维噪声高斯混合分类中的作用

• 发表时间: 2020-07
• 期刊: Thirty-seventh International Conference on Machine Learning (ICML
• 作者: Mignacco, Francesca;Krzakala, Florent;Lu, Yue M;Zdeborová, Lenka
• 通讯作者: Zdeborová, Lenka