Collaborative Research: Theoretical and Algorithmic Foundations of Variational Bayesian Inference

合作研究：变分贝叶斯推理的理论和算法基础

• 批准号: 2210717
• 负责人: Yun Yang
• 金额: $ 13.42万
• 依托单位: University of Illinois at Urbana-Champaign
• 依托单位国家: 美国
• 项目类别: Standard Grant
• 财政年份: 2022
• 资助国家: 美国
• 起止时间: 2022-07-01 至 2025-06-30
• 项目状态: 未结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=2210717&HistoricalAwards=false
• 关键词: Collaborative; Research; Theoretical; Algorithmic; Foundations

Spectacular advances in data acquisition, processing and storage techniques offer modern-day statisticians a unique opportunity to analyze large and complex datasets of unprecedented richness which arise in many scientific investigations and in studies in the social and economic fields. Bayesian inference, which combines prior knowledge and data information into a posterior distribution, provides a popular paradigm for probabilistic modeling of complex multi-level datasets and for performing associated inferential or predictive tasks in a principled fashion. For most practical problems, computing the posterior probabilities require numerical approximations; to that end, sampling-based approaches such as Markov chain Monte Carlo and deterministic approximations have both received widespread attention. Among deterministic approaches based on optimization, variational approximations, also commonly referred to as variational inference, is highly popular due to its scalability to large datasets. Through this project, the investigators will explore statistical and algorithmic properties of popular variational procedures and develop new methodology and computational tools grounded on a strong theoretical foundation. The results are targeted to empower practitioners with a better understanding of situations where variational inference is likely to be successful and where potential pitfalls exist. The research will be disseminated through articles and talks at prominent outlets. Additionally, software packages for the methods developed will be made available publicly. The investigators are committed to enhancing the pedagogical component of the proposal through advising students and developing graduate and undergraduate topic courses at their respective institutions.Motivated by the increasing need to mitigate scalability issues in Bayesian computation, variational inference has tremendously grown in popularity over the last two decades as an approximate Bayesian computational technique. Despite the proven empirical successes of variational inference in large complex data domains, systematic investigations into its statistical properties have commenced only recently. Through this project, the investigators will pose a number of foundational questions to address theoretical challenges in understanding and explaining the great empirical success of variational approximations in parameter estimation, statistical inference, and model selection, coupled with applications in novel domains. The investigators will also develop general purpose sufficient conditions to certify convergence of popularly used variational algorithms. The theoretical development will employ tools from dynamical systems, functional optimization, and optimal transport, leading to a unified treatment of statistical and algorithmic aspects of variational inference. In light of this new theory, the investigators will develop modifications to existing algorithms with certifiably better convergence behaviors.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

数据获取，处理和存储技术的壮观进步为现代统计学家提供了一个独特的机会，可以分析大型且复杂的富裕数据集，这些数据集在许多科学研究和社会和经济领域的研究中都产生。 将先验知识和数据信息与后验分布相结合的贝叶斯推论为复杂多级数据集的概率建模以及以原则上的方式执行相关的推论或预测任务提供了流行的范式。对于大多数实际问题，计算后验概率需要数值近似值；为此，基于抽样的方法，例如马尔可夫链蒙特卡洛和确定性近似值，都受到了广泛的关注。在基于优化的确定性方法中，由于其对大型数据集的可扩展性，变异近似值（通常也称为变异推断）非常受欢迎。通过该项目，研究人员将探索流行变异程序的统计和算法属性，并开发基于强大理论基础的新方法和计算工具。结果旨在使从业者能够更好地理解变异推理可能成功并存在潜在陷阱的情况。这项研究将通过文章和在著名渠道进行会谈来传播。此外，将公开提供开发方法的软件包。调查人员致力于通过为学生提供建议，发展研究生和本科主题课程来增强提案的教学成分，并在贝叶斯计算中减轻可伸缩性问题的需求增加，变异推论的越来越大，在最后的推论中，越来越多。二十年是近似贝叶斯计算技术。尽管在大型复杂数据域中证明了变异推断的经验成功，但对其统计特性的系统研究才开始最近才开始。通过该项目，研究人员将提出许多基本问题，以解决理解和解释参数估计，统计推断和模型选择方面近似值的巨大经验成功的理论挑战，再加上新颖域中的应用。研究人员还将开发出足够的条件，以证明普遍使用的变分算法的收敛性。理论发展将采用动态系统，功能优化和最佳运输的工具，从而导致对变异推理的统计和算法方面的统一处理。鉴于这一新理论，研究人员将对具有更好的融合行为的现有算法进行修改。该奖项反映了NSF的法定任务，并被认为是值得通过基金会的知识分子优点和更广泛影响的评估评估的评估来审查标准的。