CIF: Small: Exploring and Exploiting the Universality Phenomenon in High-Dimensional Estimation

CIF：小：探索和利用高维估计中的普遍性现象

• 批准号: 1910410
• 负责人: Yue Lu
• 金额: $ 49.97万
• 依托单位: Harvard University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2019
• 资助国家: 美国
• 起止时间: 2019-10-01 至 2022-09-30
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1910410&HistoricalAwards=false
• 关键词: CIF; Small; Exploring; Exploiting; Universality; CIF; Small; Exploring; Exploiting; Universality

Understanding the performance of various algorithms used in practice is a central question in information processing and machine learning. Such performance guarantees are very important to practitioners. For example, data analysts need to know how many data samples to collect for a given inference algorithm to reach a prediction with sufficient statistical accuracy and confidence. Although significant progress has been made in precisely characterizing the performance of various estimation and inference algorithms, a big gap exists between theory and practice. Most of existing theoretical work on performance analysis relies upon strong and often unrealistic assumptions on the underlying statistical models. Such idealistic models, while useful and convenient for mathematical proofs, often do not fit the situations encountered in practice. This project aims to narrow the gap between theory and practice in performance analysis by leveraging the universality phenomenon. In short, universality is the observation that universal laws govern the macroscopic behavior of many high-dimensional systems, irrespectively of how different they might be in their microscopic constructions. By exploiting the universality phenomenon, this project contributes to an understanding of the fundamental limits of various estimation and inference methods under more realistic models. In addition, this project makes broad impacts through the dissemination of datasets, the organization of workshops/tutorials, and the mentoring and supporting of students from diverse backgrounds.The specific goals of this project are organized into three main thrusts. In the first thrust, the investigator analyzes the exact asymptotic performance of a class of spectral methods that have been widely used in recent work on nonconvex optimization approaches for signal estimation. In particular, the project extends the current analysis from independent ensembles to more general ensembles, and explores new applications including multiplexed imaging and the training of multilayer neural networks. In the second thrust, the project investigates the performance bounds for regularized M-estimators when the sensing matrices are drawn from general non-independent ensembles. The third thrust of the project uses extensive numerical simulations to explore the strength, robustness, as well as the limitations of the universality phenomenon in high-dimensional estimation. The numerical experiments are guided by the theory and insights developed in the first two thrusts.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

了解实践中使用的各种算法的性能是信息处理和机器学习中的一个核心问题。这种绩效保证对从业者非常重要。例如，数据分析师需要知道为给定推理算法收集多少数据样本，以实现足够的统计准确性和信心来达到预测。尽管精确地表征各种估计和推理算法的性能方面取得了重大进展，但理论与实践之间存在很大的差距。关于绩效分析的大多数现有理论工作都依赖于对基本统计模型的强烈且通常是不切实际的假设。这种理想主义模型虽然对数学证明有用且方便，但通常不符合实践中遇到的情况。该项目旨在通过利用普遍性现象来缩小绩效分析中理论与实践之间的差距。简而言之，普遍性是普遍的观察，即普遍法律控制着许多高维系统的宏观行为，无论它们在微观构造中可能有何不同。通过利用普遍性现象，该项目有助于理解在更现实的模型下的各种估计和推理方法的基本限制。此外，该项目通过传播数据集，研讨会/教程的组织以及来自不同背景的学生的指导和支持。该项目的具体目标被组织为三个主要推力。在第一个推力中，研究者分析了一类光谱方法的确切渐近性能，这些光谱方法已在最近的非凸优化方法中广泛使用，以进行信号估计。特别是，该项目将当前的分析从独立的合奏扩展到更通用的合奏，并探索包括多路复用成像和多层神经网络培训的新应用程序。在第二个推力中，该项目研究了从一般的非独立集合中绘制传感矩阵时的正则M估计器的性能界限。该项目的第三个推力使用广泛的数值模拟来探索高维估计中的强度，鲁棒性以及普遍性现象的局限性。数值实验以前两个推力中发展的理论和见解为指导。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子优点和更广泛的影响评估标准通过评估来支持的。