Collaborative Research: CIF: Medium: Learning and Inference in High-Dimensional Models: Rigorous Analysis and Applications

合作研究：CIF：中：高维模型中的学习和推理：严谨的分析和应用

基本信息

批准号：
1955732
负责人：
Alyson Fletcher
金额：
$ 45万
依托单位：
University of California-Los Angeles
依托单位国家：
美国
项目类别：
Continuing Grant
财政年份：
2020
资助国家：
美国
起止时间：
2020-07-01 至 2024-06-30
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1955732&HistoricalAwards=false
关键词：
Collaborative Research CIF Medium Learning

项目摘要

A key feature of contemporary signal processing and machine learning problems is their massive scale. Signal processing tasks routinely involve images and videos with millions of pixels, and modern deep-learning methods often involve millions of tunable parameters. Although recent methods, particularly deep learning, have had tremendous practical success in the high-dimensional setting, they are difficult to explain from a theoretical perspective. This project seeks to develop mathematical tools to better understand such estimation and learning problems along the following directions: How does one tractably formulate precise, high-dimensional analyses of contemporary problems; what do those analyses say about the information-theoretic limits of estimation and learning; and how can these limits be approached by practical algorithms? To achieve broader impacts, the project includes dissemination in workshops, coordination with the growing machine learning industry and the development of a new module on data science to be provided to high school teachers.The project builds on the powerful approximate message passing (AMP) framework, an estimation methodology that offers the potential for a rigorous analytic understanding of modern, high-dimensional problems. Since its origin as a method for understanding linear inverse problems in compressed sensing, AMP has had tremendous success in a wide range of estimation and learning tasks. This project aims to extend the AMP framework to contemporary, large-scale learning tasks. The project is organized into three main thrusts: 1) inference with structured bilinear models, 2) learning of multi-layer neural networks, and 3) analysis with Fourier and convolutional operators. In each thrust, the project will develop fundamental mathematical theory and validate the theory on key applications, particular in image processing and statistical learning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

当代信号处理和机器学习问题的一个关键特征是它们的规模。信号处理任务通常涉及数百万像素的图像和视频，而现代的深度学习方法通常涉及数百万可调参数。尽管最近的方法，尤其是深度学习，在高维环境中取得了巨大的实践成功，但从理论的角度来看，它们很难解释。该项目旨在开发数学工具，以更好地理解以下方向的估计和学习问题：一个人如何对当代问题进行精确的高维分析；这些分析对估计和学习的信息理论限制有何评论？以及如何通过实用算法处理这些限制？为了实现更广泛的影响，该项目包括在研讨会上的传播，与不断发展的机器学习行业的协调以及将向高中教师提供的新模块的开发。该项目以强大的近似信息传递（AMP）框架为基础，这是一种估算方法，该方法为对现代，高维问题进行严格的分析理解提供了潜力。由于其起源是理解压缩感应中线性反向问题的方法，因此AMP在广泛的估计和学习任务中取得了巨大的成功。该项目旨在将AMP框架扩展到当代大规模学习任务。该项目分为三个主要推力：1）与结构化双线性模型的推断，2）学习多层神经网络以及3）与傅立叶和卷积操作员分析。在每个推力中，该项目将发展基本的数学理论，并验证有关关键应用的理论，尤其是在图像处理和统计学习中。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的智力优点评估和更广泛的影响来获得支持的。

项目成果

期刊论文数量（8）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

Generalized Autoregressive Linear Models for Discrete High-Dimensional Data

离散高维数据的广义自回归线性模型

DOI：
10.1109/jsait.2020.3041714
发表时间：
2020
期刊：
IEEE Journal on Selected Areas in Information Theory
影响因子：
0
作者：
Pandit, Parthe;Sahraee-Ardakan, Mojtaba;Amini, Arash A.;Rangan, Sundeep;Fletcher, Alyson K.
通讯作者：
Fletcher, Alyson K.

Implicit Bias of Linear RNNs

DOI：
发表时间：
2021-01
期刊：
The Annals of Statistics
影响因子：
0
作者：
M Motavali Emami;Mojtaba Sahraee-Ardakan;Parthe Pandit;S. Rangan;A. Fletcher
通讯作者：
M Motavali Emami;Mojtaba Sahraee-Ardakan;Parthe Pandit;S. Rangan;A. Fletcher

Asymptotics of Ridge Regression in Convolutional Models

DOI：
发表时间：
2021-03
期刊：
ArXiv
影响因子：
0
作者：
Mojtaba Sahraee-Ardakan;Tung Mai;Anup B. Rao;Ryan A. Rossi;S. Rangan;A. Fletcher
通讯作者：
Mojtaba Sahraee-Ardakan;Tung Mai;Anup B. Rao;Ryan A. Rossi;S. Rangan;A. Fletcher

Inference With Deep Generative Priors in High Dimensions

高维深度生成先验的推理

DOI：
10.1109/jsait.2020.2986321
发表时间：
2020
期刊：
IEEE Journal on Selected Areas in Information Theory
影响因子：
0
作者：
Pandit, Parthe;Sahraee-Ardakan, Mojtaba;Rangan, Sundeep;Schniter, Philip;Fletcher, Alyson K.
通讯作者：
Fletcher, Alyson K.

Instability and Local Minima in GAN Training with Kernel Discriminators