Sampling from Distributions with Intractable Integrals

从具有棘手积分的分布中采样

• 批准号: 1007457
• 负责人: Faming Liang
• 金额: $ 10万
• 依托单位: Texas A&M Research Foundation
• 依托单位国家: 美国
• 项目类别: Continuing Grant
• 财政年份: 2010
• 资助国家: 美国
• 起止时间: 2010-08-01 至 2013-07-31
• 项目状态: 已结题
• 来源: https://www.nsf.gov/awardsearch/showAward?AWD_ID=1007457&HistoricalAwards=false
• 关键词: Sampling; Distributions; Intractable; Integrals

During the past five decades, Markov chain Monte Carlo (MCMC) methods have been developed as a versatile and powerful tool for scientific computing. However, as known by many researchers, conventional MCMC methods suffer from the inability to sample from distributions with intractable integrals. The goal of this project is to develop some innovative Monte Carlo algorithms which are capable of sampling from distributions with intractable integrals. To achieve this goal, the PI proposes a new population Monte Carlo algorithm---Monte Carlo dynamically weighted importance sampling (MCDWIS). In simulations, MCDWIS replaces the ratio of intractable integrals by its Monte Carlo estimate, and the bias introduced thereby is counterbalanced by giving different weights to new samples produced. MCDWIS allows for the use of Monte Carlo estimates in MCMC simulations, while leaving the target distribution invariant with respect to important weights. Unlike auxiliary variable MCMC methods, MCDWIS avoids the requirement for perfect samples, and thus can be applied to many statistical models for which perfect sampling is unavailable or very expensive. As discussed in the proposal, MCDWIS can also be used to sample from incomplete posterior distributions for missing data and random effects-related models (e.g., generalized linear mixed models), which are traditionally treated with the expectation-maximization (EM) or Monte Carlo EM algorithms. In addition to providing a fully Bayesian analysis for these models, the MCDWIS can potentially overcome, due to its self-adjusting mechanism, the local-trap problem suffered by the EM and Monte Carlo EM algorithms. In this proposal, the PI also proposes an importance sampling-targeted stochastic approximation Monte Carlo algorithm, the so-called importance stochastic approximation Monte Carlo algorithm, which can be used for Bayesian inference for the models with intractable normalizing constants.The intellectual merit of this project is to provide some innovative computational methods, which are expected to play a major role in statistical inference for an important class of scientific models, including random graph models used in social network analysis, autonormal models used in spatial data analysis, autologistic models used in disease mapping, and generalized linear mixed models used in biomedical data analysis, among others. Successful inferences of the models will enhance people's underderstanding to the underlying natural, social, or biological systems. This project will have broader impacts in both communities of statistical methodology and scientific computing. The research results will be disseminated to these communities via direct collaboration with researchers in other disciplines, conference presentations, books, and papers to be published in academic journals. The project will have also significant impacts on education through direct involvement of graduate students in the project and incorporation of results into undergraduate and graduate courses.

在过去的五年中，马尔可夫链蒙特卡罗 (MCMC) 方法已发展成为一种多功能且强大的科学计算工具。 然而，正如许多研究人员所知，传统的 MCMC 方法无法从具有棘手积分的分布中进行采样。该项目的目标是开发一些创新的蒙特卡罗算法，这些算法能够从具有棘手积分的分布中进行采样。为了实现这一目标，PI提出了一种新的总体蒙特卡罗算法——蒙特卡罗动态加权重要性采样（MCDWIS）。在模拟中，MCDWIS 用蒙特卡罗估计代替了难以处理的积分的比率，并且通过对产生的新样本赋予不同的权重来抵消由此引入的偏差。 MCDWIS 允许在 MCMC 模拟中使用蒙特卡罗估计，同时使目标分布相对于重要权重保持不变。与辅助变量 MCMC 方法不同，MCDWIS 避免了对完美样本的要求，因此可以应用于许多完美采样不可用或非常昂贵的统计模型。正如提案中所讨论的，MCDWIS 还可用于从缺失数据和随机效应相关模型（例如广义线性混合模型）的不完整后验分布中进行采样，这些模型传统上使用期望最大化 (EM) 或蒙特卡罗进行处理EM 算法。除了为这些模型提供完整的贝叶斯分析之外，MCDWIS 由于其自我调整机制，还可以潜在地克服 EM 和蒙特卡罗 EM 算法所遇到的局部陷阱问题。在该提案中，PI还提出了一种以重要性采样为目标的随机逼近蒙特卡罗算法，即所谓的重要性随机逼近蒙特卡罗算法，该算法可用于具有棘手归一化常数的模型的贝叶斯推理。该项目旨在提供一些创新的计算方法，预计将在一类重要的科学模型的统计推断中发挥重要作用，包括社交网络分析中使用的随机图模型、空间分析中使用的自正态模型数据分析、疾病绘图中使用的自逻辑模型以及生物医学数据分析中使用的广义线性混合模型等。模型的成功推理将增强人们对潜在的自然、社会或生物系统的理解。该项目将对统计方法和科学计算界产生更广泛的影响。研究成果将通过与其他学科研究人员的直接合作、会议演讲、书籍和在学术期刊上发表的论文来传播给这些社区。 该项目还将通过研究生直接参与该项目并将成果纳入本科生和研究生课程，对教育产生重大影响。