# On Russian Roulette Estimates for Bayesian Inference with Doubly-Intractable Likelihoods

arXiv: https://arxiv.org/abs/1306.4032 · HTML: https://mlsanity.com/paper/1306.4032/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Jun 2013.

## Built on by (most cited first, 16 of 16)

- [Modern Bayesian Experimental Design](https://mlsanity.com/paper/2302.14545/) (2023)
- [Robust Generalised Bayesian Inference for Intractable Likelihoods](https://mlsanity.com/paper/2104.07359/) (2021)
- [Meta-Learning Reliable Priors in the Function Space](https://mlsanity.com/paper/2106.03195/) (2021)
- [Bias-Free Scalable Gaussian Processes via Randomized Truncations](https://mlsanity.com/paper/2102.06695/) (2021)
- [Generalised Bayesian Inference for Discrete Intractable Likelihood](https://mlsanity.com/paper/2206.08420/) (2022)
- [Entropy-based adaptive Hamiltonian Monte Carlo](https://mlsanity.com/paper/2110.14625/) (2021)
- [Multi-fidelity Monte Carlo: a pseudo-marginal approach](https://mlsanity.com/paper/2210.01534/) (2022)
- [Accelerating Look-ahead in Bayesian Optimization: Multilevel Monte Carlo is All you Need](https://mlsanity.com/paper/2402.02111/) (2024)
- [Improved Stochastic Optimization of LogSumExp](https://mlsanity.com/paper/2509.24894/) (2025)
- [On MCMC for variationally sparse Gaussian processes: A pseudo-marginal approach](https://mlsanity.com/paper/2103.03321/) (2021)
- [Spectral Regularization: an Inductive Bias for Sequence Modeling](https://mlsanity.com/paper/2211.02255/) (2022)
- [Leveraging Nested MLMC for Sequential Neural Posterior Estimation with Intractable Likelihoods](https://mlsanity.com/paper/2401.16776/) (2024)
- [Sampling as Bandits: Evaluation-Efficient Design for Black-Box Densities](https://mlsanity.com/paper/2509.01437/) (2025)
- [Exact and Approximate MCMC for Doubly-intractable Probabilistic Graphical Models Leveraging the Underlying Independence Model](https://mlsanity.com/paper/2510.03587/) (2025)
- [BayesSum: Bayesian Quadrature in Discrete Spaces](https://mlsanity.com/paper/2512.16105/) (2025)
- [Robust Bayesian Inference for Unnormalized Models with Mixed-Domain Data](https://mlsanity.com/paper/2609.01783/) (2026)

## Abstract (arXiv)

A large number of statistical models are "doubly-intractable": the likelihood normalising term, which is a function of the model parameters, is intractable, as well as the marginal likelihood (model evidence). This means that standard inference techniques to sample from the posterior, such as Markov chain Monte Carlo (MCMC), cannot be used. Examples include, but are not confined to, massive Gaussian Markov random fields, autologistic models and Exponential random graph models. A number of approximate schemes based on MCMC techniques, Approximate Bayesian computation (ABC) or analytic approximations to the posterior have been suggested, and these are reviewed here. Exact MCMC schemes, which can be applied to a subset of doubly-intractable distributions, have also been developed and are described in this paper. As yet, no general method exists which can be applied to all classes of models with doubly-intractable posteriors. In addition, taking inspiration from the Physics literature, we study an alternative method based on representing the intractable likelihood as an infinite series. Unbiased estimates of the likelihood can then be obtained by finite time stochastic truncation of the series via Russian Roulette sampling, although the estimates are not necessarily positive. Results from the Quantum Chromodynamics literature are exploited to allow the use of possibly negative estimates in a pseudo-marginal MCMC scheme such that expectations with respect to the posterior distribution are preserved. The methodology is reviewed on well-known examples such as the parameters in Ising models, the posterior for Fisher-Bingham distributions on the d-Sphere and a large-scale Gaussian Markov Random Field model describing the Ozone Column data. This leads to a critical assessment of the strengths and weaknesses of the methodology with pointers to ongoing research.