# Noisy Monte Carlo: Convergence of Markov chains with approximate transition kernels

arXiv: https://arxiv.org/abs/1403.5496 · HTML: https://mlsanity.com/paper/1403.5496/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Mar 2014.

## Built on by (most cited first, 10 of 10)

- [A Survey of Monte Carlo Methods for Parameter Estimation](https://mlsanity.com/paper/2107.11820/) (2021)
- [A survey of Monte Carlo methods for noisy and costly densities with application to reinforcement learning and ABC](https://mlsanity.com/paper/2108.00490/) (2021)
- [Estimation and Inference in Distributional Reinforcement Learning](https://mlsanity.com/paper/2309.17262/) (2023)
- [Maximum Likelihood Learning of Unnormalized Models for Simulation-Based Inference](https://mlsanity.com/paper/2210.14756/) (2022)
- [Approximate Bayesian inference from noisy likelihoods with Gaussian process emulated MCMC](https://mlsanity.com/paper/2104.03942/) (2021)
- [Empirical evaluation of normalizing flows in Markov Chain Monte Carlo](https://mlsanity.com/paper/2412.17136/) (2024)
- [Gaussian Processes to speed up MCMC with automatic exploratory-exploitation effect](https://mlsanity.com/paper/2109.13891/) (2021)
- [Approximate Gibbs Sampler for Efficient Inference of Hierarchical Bayesian Models for Grouped Count Data](https://mlsanity.com/paper/2211.15771/) (2022)
- [Clustered Mallows Model](https://mlsanity.com/paper/2403.12880/) (2024)
- [Exact and Approximate MCMC for Doubly-intractable Probabilistic Graphical Models Leveraging the Underlying Independence Model](https://mlsanity.com/paper/2510.03587/) (2025)

## Abstract (arXiv)

Monte Carlo algorithms often aim to draw from a distribution π by simulating a Markov chain with transition kernel P such that π is invariant under P. However, there are many situations for which it is impractical or impossible to draw from the transition kernel P. For instance, this is the case with massive datasets, where is it prohibitively expensive to calculate the likelihood and is also the case for intractable likelihood models arising from, for example, Gibbs random fields, such as those found in spatial statistics and network analysis. A natural approach in these cases is to replace P by an approximation hatP. Using theory from the stability of Markov chains we explore a variety of situations where it is possible to quantify how 'close' the chain given by the transition kernel hatP is to the chain given by P. We apply these results to several examples from spatial statistics and network analysis.