# Robust Generalised Bayesian Inference for Intractable Likelihoods

arXiv: https://arxiv.org/abs/2104.07359 · HTML: https://mlsanity.com/paper/2104.07359/

## Key facts

- Cited by 53 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 111 references that are in the map.
- First posted on arXiv 15 Apr 2021.

## Builds on (most cited first, 30 of 34)

- [Automatic differentiation in machine learning: a survey](https://mlsanity.com/paper/1502.05767/) (2015)
- [A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluation](https://mlsanity.com/paper/1602.03253/) (2016)
- [A General Framework for Updating Belief Distributions](https://mlsanity.com/paper/1306.6430/) (2013)
- [A Complete Recipe for Stochastic Gradient MCMC](https://mlsanity.com/paper/1506.04696/) (2015)
- [A Kernel Test of Goodness of Fit](https://mlsanity.com/paper/1602.02964/) (2016)
- [On the properties of variational approximations of Gibbs posteriors](https://mlsanity.com/paper/1506.04091/) (2015)
- [Measuring Sample Quality with Kernels](https://mlsanity.com/paper/1703.01717/) (2017)
- [Inconsistency of Bayesian Inference for Misspecified Linear Models, and a Proposal for Repairing It](https://mlsanity.com/paper/1412.3730/) (2014)
- [Measuring Sample Quality with Stein's Method](https://mlsanity.com/paper/1506.03039/) (2015)
- [Generalized Variational Inference: Three arguments for deriving new Posteriors](https://mlsanity.com/paper/1904.02063/) (2019)
- [Minimum Stein Discrepancy Estimators](https://mlsanity.com/paper/1906.08283/) (2019)
- [Assigning a value to a power likelihood in a general Bayesian model](https://mlsanity.com/paper/1701.08515/) (2017)
- [Learning deep kernels for exponential family densities](https://mlsanity.com/paper/1811.08357/) (2018)
- [MMD-Bayes: Robust Bayesian Estimation via Maximum Mean Discrepancy](https://mlsanity.com/paper/1909.13339/) (2019)
- [General Bayesian Updating and the Loss-Likelihood Bootstrap](https://mlsanity.com/paper/1709.07616/) (2017)
- [Blindness of score-based methods to isolated components and mixing proportions](https://mlsanity.com/paper/2008.10087/) (2020)
- [Robust Bayesian inference via coarsening](https://mlsanity.com/paper/1506.06101/) (2015)
- [Stein Variational Gradient Descent With Matrix-Valued Kernels](https://mlsanity.com/paper/1910.12794/) (2019)
- [Stochastic Stein Discrepancies](https://mlsanity.com/paper/2007.02857/) (2020)
- [Stein Point Markov Chain Monte Carlo](https://mlsanity.com/paper/1905.03673/) (2019)
- [On Graphical Models via Univariate Exponential Family Distributions](https://mlsanity.com/paper/1301.4183/) (2013)
- [Optimal Thinning of MCMC Output](https://mlsanity.com/paper/2005.03952/) (2020)
- [Measuring Sample Quality with Diffusions](https://mlsanity.com/paper/1611.06972/) (2016)
- [Random Feature Stein Discrepancies](https://mlsanity.com/paper/1806.07788/) (2018)
- [Sliced Kernelized Stein Discrepancy](https://mlsanity.com/paper/2006.16531/) (2020)
- [A comparison of learning rate selection methods in generalized Bayesian inference](https://mlsanity.com/paper/2012.11349/) (2020)
- [On Russian Roulette Estimates for Bayesian Inference with Doubly-Intractable Likelihoods](https://mlsanity.com/paper/1306.4032/) (2013)
- [Gradient-free Hamiltonian Monte Carlo with Efficient Kernel Exponential Families](https://mlsanity.com/paper/1506.02564/) (2015)
- [Objective Bayesian inference with proper scoring rules](https://mlsanity.com/paper/1711.10819/) (2017)
- [Fisher Efficient Inference of Intractable Models](https://mlsanity.com/paper/1805.07454/) (2018)

## Built on by (most cited first, 30 of 53)

- [Detecting Model Misspecification in Amortized Bayesian Inference with Neural Networks](https://mlsanity.com/paper/2112.08866/) (2021)
- [Robust Bayesian Inference for Simulator-based Models via the MMD Posterior Bootstrap](https://mlsanity.com/paper/2202.04744/) (2022)
- [A Rigorous Link between Deep Ensembles and (Variational) Bayesian Methods](https://mlsanity.com/paper/2305.15027/) (2023)
- [A Trust Crisis In Simulation-Based Inference? Your Posterior Approximations Can Be Unfaithful](https://mlsanity.com/paper/2110.06581/) (2021)
- [Towards Reliable Simulation-Based Inference with Balanced Neural Ratio Estimation](https://mlsanity.com/paper/2208.13624/) (2022)
- [Addressing Misspecification in Simulation-based Inference through Data-driven Calibration](https://mlsanity.com/paper/2405.08719/) (2024)
- [Outlier-robust Kalman Filtering through Generalised Bayes](https://mlsanity.com/paper/2405.05646/) (2024)
- [Adversarial robustness of amortized Bayesian inference](https://mlsanity.com/paper/2305.14984/) (2023)
- [Generalized Bayesian Inference for Scientific Simulators via Amortized Cost Estimation](https://mlsanity.com/paper/2305.15208/) (2023)
- [Composite Goodness-of-fit Tests with Kernels](https://mlsanity.com/paper/2111.10275/) (2021)
- [Robust and Scalable Bayesian Online Changepoint Detection](https://mlsanity.com/paper/2302.04759/) (2023)
- [On the Robustness to Misspecification of α-Posteriors and Their Variational Approximations](https://mlsanity.com/paper/2104.08324/) (2021)
- [Targeted Separation and Convergence with Kernel Discrepancies](https://mlsanity.com/paper/2209.12835/) (2022)
- [Generalised Bayesian Inference for Discrete Intractable Likelihood](https://mlsanity.com/paper/2206.08420/) (2022)
- [Robust and Conjugate Gaussian Process Regression](https://mlsanity.com/paper/2311.00463/) (2023)
- [Controlling Moments with Kernel Stein Discrepancies](https://mlsanity.com/paper/2211.05408/) (2022)
- [Sourcerer: Sample-based Maximum Entropy Source Distribution Estimation](https://mlsanity.com/paper/2402.07808/) (2024)
- [Robust and Conjugate Spatio-Temporal Gaussian Processes](https://mlsanity.com/paper/2502.02450/) (2025)
- [Simulation-based Bayesian inference under model misspecification](https://mlsanity.com/paper/2503.12315/) (2025)
- [A Dictionary of Closed-Form Kernel Mean Embeddings](https://mlsanity.com/paper/2504.18830/) (2025)
- [Misspecification-robust Sequential Neural Likelihood for Simulation-based Inference](https://mlsanity.com/paper/2301.13368/) (2023)
- [Detecting Model Misspecification in Amortized Bayesian Inference with Neural Networks: An Extended Investigation](https://mlsanity.com/paper/2406.03154/) (2024)
- [Biological Sequence Kernels with Guaranteed Flexibility](https://mlsanity.com/paper/2304.03775/) (2023)
- [Concentration properties of fractional posterior in 1-bit matrix completion](https://mlsanity.com/paper/2404.08969/) (2024)
- [On the Robustness of Kernel Goodness-of-Fit Tests](https://mlsanity.com/paper/2408.05854/) (2024)
- [A sparse PAC-Bayesian approach for high-dimensional quantile prediction](https://mlsanity.com/paper/2409.01687/) (2024)
- [Using Perturbation to Improve Goodness-of-Fit Tests based on Kernelized Stein Discrepancy](https://mlsanity.com/paper/2304.14762/) (2023)
- [Scalable Valuation of Human Feedback through Provably Robust Model Alignment](https://mlsanity.com/paper/2505.17859/) (2025)
- [A Fourier representation of kernel Stein discrepancy with application to Goodness-of-Fit tests for measures on infinite dimensional Hilbert spaces](https://mlsanity.com/paper/2206.04552/) (2022)
- [PAC-Bayesian Soft Actor-Critic Learning](https://mlsanity.com/paper/2301.12776/) (2023)

## Abstract (arXiv)

Generalised Bayesian inference updates prior beliefs using a loss function, rather than a likelihood, and can therefore be used to confer robustness against possible mis-specification of the likelihood. Here we consider generalised Bayesian inference with a Stein discrepancy as a loss function, motivated by applications in which the likelihood contains an intractable normalisation constant. In this context, the Stein discrepancy circumvents evaluation of the normalisation constant and produces generalised posteriors that are either closed form or accessible using standard Markov chain Monte Carlo. On a theoretical level, we show consistency, asymptotic normality, and bias-robustness of the generalised posterior, highlighting how these properties are impacted by the choice of Stein discrepancy. Then, we provide numerical experiments on a range of intractable distributions, including applications to kernel-based exponential family models and non-Gaussian graphical models.