Mean-Field Models in Statistics

统计学中的平均场模型

• 批准号: 2113414
• 负责人: Sumit Mukherjee
• 金额: $ 17万
• 依托单位: Columbia University
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2021
• 资助国家: 美国
• 起止时间: 2021-07-01 至 2025-06-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2113414&HistoricalAwards=false
• 关键词: Mean; Field; Models; Statistics

Frequently in high dimensional Bayesian models, the posterior distribution is complicated and exhibits non-trivial dependence. Understanding the behavior of such models, both theoretically and empirically, is a challenge. In these situations, variational inference provides an efficient and scalable method of inference, when more traditional methods based on Markov Chain Monte Carlo are computationally prohibitive. One of the most common techniques for variational inference is the naive mean field approximation. However, despite its wide usage, not much is known about rigorous guarantees for the naive mean field method. This project aims to address this question, by studying the validity of naive mean field methods for several examples of interest.This project defines a formal notion of correctness of the naive mean field approximation, which requires that the leading order of the log normalizing constant of the high dimensional distribution is predicted correctly by the mean field prediction formula. This definition does not require the high dimensional distribution of interest to arise as a posterior distribution. If the naive mean field approximation is indeed asymptotically correct, the high dimensional distribution of interest should be well approximated by a mixture of product distributions. If further, the optimization in the mean field prediction formula has a unique maximizer, then essentially the high dimensional distribution should be close to a product measure, and it is natural enquire about Law of Large Numbers, Concentration, Fluctuations, and Asymptotic Properties of Estimators. The PI plans to study these questions for three concrete examples: (a) (Bayesian) Linear Regression, (b) (Bayesian) Mixture of Gaussians, and (c) Exponential Random Graph Models.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

通常在高维贝叶斯模型中，后部分布很复杂，并且表现出非平凡的依赖性。理论上和经验上了解这种模型的行为是一个挑战。在这些情况下，当基于马尔可夫链蒙特卡洛的更传统的方法在计算上，变异推理提供了一种有效且可扩展的推理方法。变异推断的最常见技术之一是幼稚的平均场近似。但是，尽管使用广泛，但对幼稚平均野外方法的严格保证并不多。该项目的目的是通过研究幼稚平均场方法对几个感兴趣的示例的有效性来解决这个问题。本项目定义了幼稚平均场近似正确性的正式概念，这要求通过平均野外预测公式正确预测高尺寸分布的对数正常化的正常化顺序。该定义不需要高维分布作为后验分布。如果幼稚的平均场近似确实在渐近上正确，则应通过产物分布的混合物很好地近似兴趣的高维分布。如果进一步，平均场预测公式中的优化具有独特的最大化器，则本质上，高维分布应接近产品度量，并且自然要查询估计量的大量，浓度，波动和渐近性能的定律。 PI计划在三个具体示例中研究这些问题：（a）（贝叶斯）线性回归，（b）（贝叶斯）高斯人的混合物以及（c）指数的随机图模型。该奖项反映了NSF的法定任务，并且通过评估该基金会的知识绩效和广泛的影响，认为NSF的法定任务值得通过评估值得进行评估。