AF: Medium: Collaborative Research: Theoretical Foundations of Deep Generative Models and High-Dimensional Distributions

AF：中：协作研究：深度生成模型和高维分布的理论基础

基本信息

批准号：
1901281
负责人：
Georgios-Alex Dimakis
金额：
$ 49.99万
依托单位：
University of Texas at Austin
依托单位国家：
美国
项目类别：
Continuing Grant
财政年份：
2019
资助国家：
美国
起止时间：
2019-07-01 至 2024-06-30
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1901281&HistoricalAwards=false
关键词：
AF Medium Collaborative Research Theoretical

项目摘要

Current technology is driving our ability to collect, store and process data at an unprecedented scale. Ranging from image, audio and video to social-network, medical and biological datasets, modern applications require us to model and reason about complex data over extremely large domains. It is well-known, however, that this cannot be done in a rigorous manner unless simplifying assumptions can be made about how the data of interest are generated. Accordingly, a long line of investigation in Probability Theory, Statistical Physics, Information Theory and Machine Learning has been preoccupied with developing mathematical and algorithmic frameworks that allow for succinct representation and inference of high-dimensional distributions with simplifying structure. This project will go beyond the standard frameworks in these fields to advance the theoretical foundations of a research frontier that has recently emerged as a promising approach towards a more accurate modeling of high-dimensional data. In particular, this project will study the theoretical foundations of learning, testing and statistical inference of high-dimensional data that are generated by deep neural network-based generative models, developing mathematically rigorous quality guarantees, which is a big desideratum in the field of deep learning. On the practical front, this work has the potential to significantly improve the performance of image-reconstruction algorithms compared to state-of-the-art, and therefore to have significant impact on various applications of image reconstruction such as rapid magnetic resonance imaging (MRI).Since the introduction of deep neural network-based generative models, there have been numerous approaches for how to architect them, how to train them using samples from a distribution of interest, and how to use them for downstream inference tasks; these have delivered impressive practical results. On the other hand, there has also been a lot of debate around the quality of deep generative models that are trained via current techniques, and it has been recognized that there are significant challenges in optimizing, evaluating and scaling the dimensionality of deep generative models, as well as in using them for data recovery. This project develops three research thrusts targeting these challenges, namely: (i) developing better algorithms for training deep generative models, and for using these models as "regularizers" in signal-processing applications; (ii) developing statistical techniques for evaluating the quality of a deep generative model against the distribution whose samples it was trained on; (iii) proposing architectures and algorithms for scaling up the dimensionality of deep generating models while providing statistical accuracy guarantees. This work will rely on techniques from non-convex and combinatorial optimization, signal processing, game theory, high-dimensional statistics, and statistical physics, and build connections between these fields and deep learning.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

当前的技术正在推动我们以前所未有的规模收集，存储和处理数据的能力。从图像，音频和视频到社交网络，医学和生物数据集，现代应用程序要求我们对大型域上的复杂数据进行建模和理由。但是，众所周知，除非可以对感兴趣的数据生成如何生成，否则这不能以严格的方式进行。因此，概率理论，统计物理学，信息理论和机器学习的一长串研究已被置于开发数学和算法框架上，这些框架允许使用简化结构进行简洁的表示和高维分布的推理。该项目将超越这些领域的标准框架，以推动研究边界的理论基础，该基础最近已成为一种有前途的方法，以更准确地建模高维数据。特别是，该项目将研究高维数据的学习，测试和统计推断的理论基础，这些基础是由深度神经网络的生成模型产生的，开发了数学上严格的质量保证，这是深度学习领域中的很大的质量保证。在实用方面，这项工作有可能显着提高图像重建算法的性能，从而对图像重建的各种应用（例如快速磁共振成像（MRI））具有重大影响，因此，基于深层的神经网络的生成模型，对它们进行了多种培训的方法，以培训它们的培训方式，从而使他们有多种培训的方法来培训这些模型，从而培训了这些方法，该方法是训练的方法。任务；这些取得了令人印象深刻的实际结果。另一方面，关于通过当前技术培训的深层生成模型的质量也引起了很多争论，并且已经认识到，在优化，评估和扩展深层生成模型的维度以及使用它们以进行数据恢复方面存在重大挑战。该项目开发了针对这些挑战的三项研究，即：（i）为培训深层生成模型的更好的算法，并将这些模型用作信号处理应用程序中的“正规化器”；（ii）开发统计技术，以评估深层生成模型的质量，以对其样本进行培训的分布；（iii）提出架构和算法，以扩大深层生成模型的维度，同时提供统计精度保证。这项工作将依赖于非凸和组合优化，信号处理，游戏理论，高维统计和统计物理学的技术，并在这些领域与深度学习之间建立联系。该奖项反映了NSF的法定任务，并通过基金会的知识绩效和广泛的影响来评估NSF的法定任务，并被视为值得的支持。