RI: Small: Frontiers in Monte Carlo and Variational Inference

RI：小：蒙特卡罗和变分推理的前沿

基本信息

批准号：
1908577
负责人：
Justin Domke
金额：
$ 44.96万
依托单位：
University of Massachusetts Amherst
依托单位国家：
美国
项目类别：
Standard Grant
财政年份：
2019
资助国家：
美国
起止时间：
2019-09-01 至 2023-08-31
项目状态：
已结题

来源：
https://www.nsf.gov/awardsearch/showAward?AWD_ID=1908577&HistoricalAwards=false
关键词：
RI Small Frontiers Monte Carlo

项目摘要

Probabilistic inference allows humans to gain insight and make predictions from data. There is an ever-growing need in business, government, and science to answer questions using data. Often, these questions are best answered by phrasing them as probability calculations. As data sets grow larger and more complex, probability calculations are increasingly difficult and often cannot be performed exactly within reasonable time budgets. This project will promote science and technology by providing new theoretical results, algorithms, and empirical knowledge about how to compute approximate answers to probabilistic queries in a way that achieves good tradeoffs between accuracy and efficiency. In particular, the project will study how to best combine the strengths of two different strategies for calculating probabilities. This work will provide new techniques that are practical, have tunable accuracy, and scale to very large data sets.To meet these goals, this project will combine two different approaches to probabilistic inference: variational inference (VI), and Monte Carlo (MC). MC algorithms are general-purpose and are asymptotically exact, but may fail to give good answers in reasonable time or scale large data sets. In contrast, VI is a way to get a "pretty good answer, quickly" by restricting the approximate posterior to tractable family. This project will combine these in a principled way to derive algorithms that are general-purpose, practical, have tunable accuracy, and scale to very large data sets. The new algorithms are expected to achieve time-accuracy tradeoffs that dominate Monte Carlo methods for a wide range of problems and time budgets. The proposed methods will 1) incorporate strengths of Monte Carlo methods into variational inference by designing approximating families based on Monte Carlo estimators; and 2) improve the usefulness of variational inference for downstream tasks by adapting divergences and approximating families to the needs of a downstream Monte Carlo estimator. The project will result in a comprehensive evaluation benchmark as well as a set of practical techniques to make the method more effective. A novel application in ecology will demonstrate the project's real-world potential.This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

概率推理使人类能够从数据中获得洞察力并做出预测。商业、政府和科学界越来越需要使用数据来回答问题。通常，这些问题的最佳答案是将它们表述为概率计算。随着数据集变得越来越大、越来越复杂，概率计算变得越来越困难，并且通常无法在合理的时间预算内准确执行。该项目将通过提供新的理论结果、算法和经验知识来促进科学技术的发展，这些理论结果、算法和经验知识涉及如何以在准确性和效率之间实现良好权衡的方式计算概率查询的近似答案。特别是，该项目将研究如何最好地结合两种不同策略的优势来计算概率。这项工作将提供实用、精度可调且可扩展到非常大的数据集的新技术。为了实现这些目标，该项目将结合两种不同的概率推理方法：变分推理 (VI) 和蒙特卡洛 (MC) 。 MC 算法是通用的并且渐近精确，但可能无法在合理的时间内或大规模数据集上给出良好的答案。相比之下，VI 是一种通过限制易处理家庭的近似后验来获得“快速、相当好的答案”的方法。该项目将以原则性的方式将这些结合起来，派生出通用、实用、具有可调精度并可扩展到非常大的数据集的算法。新算法预计将实现时间精度的权衡，在解决各种问题和时间预算方面，这种权衡在蒙特卡罗方法中占主导地位。所提出的方法将 1）通过设计基于蒙特卡罗估计量的近似族，将蒙特卡罗方法的优势融入到变分推理中； 2）通过调整分歧和逼近族以满足下游蒙特卡洛估计器的需求，提高变分推理对下游任务的有用性。该项目将产生一个综合评估基准以及一套实用技术，使该方法更加有效。生态学中的新颖应用将展示该项目的现实潜力。该奖项反映了 NSF 的法定使命，并通过使用基金会的智力价值和更广泛的影响审查标准进行评估，被认为值得支持。

项目成果

期刊论文数量（6）

专著数量（0）

科研奖励数量（0）

会议论文数量（0）

专利数量（0）

Variational Marginal Particle Filters

DOI：
发表时间：
2021-09
期刊：
ArXiv
影响因子：
0
作者：
Jinlin Lai;D. Sheldon;Justin Domke
通讯作者：
Jinlin Lai;D. Sheldon;Justin Domke

Amortized Variational Inference for Hierarchical Distributions

层次分布的摊销变分推理

DOI：
发表时间：
2021
期刊：
Advances in neural information processing systems
影响因子：
0
作者：
Agrawal, Abhinav;Domke, Justin
通讯作者：
Domke, Justin

MCMC Variational Inference via Uncorrected Hamiltonian Annealing

通过未修正的哈密顿退火进行 MCMC 变分推理

DOI：
发表时间：
2021
期刊：
Advances in neural information processing systems
影响因子：
0
作者：
Geffner, Tomas;Domke, Justin
通讯作者：
Domke, Justin

Variational Inference with Locally Enhanced Bounds for Hierarchical

分层局部增强界限的变分推理

DOI：
发表时间：
2022
期刊：
The international conference on machine learning
影响因子：
0
作者：
Geffner, Tomas;Domke, Justin
通讯作者：
Domke, Justin

Reconstructing bird trajectories from pressure and wind data using a highly optimized hidden Markov model

使用高度优化的隐马尔可夫模型根据压力和风数据重建鸟类轨迹

DOI：
10.1111/2041-210x.14082
发表时间：
2023
期刊：
Methods in Ecology and Evolution
影响因子：
6.6
作者：
Nussbaumer, Raphaël;Gravey, Mathieu;Briedis, Martins;Liechti, Felix;Sheldon, Daniel
通讯作者：
Sheldon, Daniel