mlsanityData to 14 Oct 2026

Approximate Bayesian inference from noisy likelihoods with Gaussian process emulated MCMC

arXiv 2104.03942 ↗First posted 8 Apr 2021stat.ME stat.CO stat.MLcited by 3

Not on the map. Why?

Abstract (arXiv)

We present a framework for approximate Bayesian inference when only a limited number of noisy log-likelihood evaluations can be obtained due to computational constraints, which is becoming increasingly common for applications of complex models. We model the log-likelihood function using a Gaussian process (GP) and the main methodological innovation is to apply this model to emulate the progression that an exact Metropolis-Hastings (MH) sampler would take if it was applicable. Informative log-likelihood evaluation locations are selected using a sequential experimental design strategy until the MH accept/reject decision is done accurately enough according to the GP model. The resulting approximate sampler is conceptually simple and sample-efficient. It is also more robust to violations of GP modelling assumptions compared with earlier, related "Bayesian optimisation-like" methods tailored for Bayesian inference. We discuss some theoretical aspects and various interpretations of the resulting approximate MH sampler, and demonstrate its benefits in the context of Bayesian and generalised Bayesian likelihood-free inference for simulator-based statistical models.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 32 of its 82 references that are in the map. First posted on arXiv 8 Apr 2021.

Builds on

32 of 82 references.

References · 32

  1. 2018A Tutorial on Bayesian Optimizationcited by 744
  2. 2019The frontier of simulation-based inferencecited by 502
  3. 2013A General Framework for Updating Belief Distributionscited by 175
  4. 2020Efficiently Sampling Functions from Gaussian Process Posteriorscited by 127
  5. 2016Likelihood-free inference by ratio estimationcited by 81
  6. 2015Bayesian Optimization for Likelihood-Free Inference of Simulator-Based Statistical Modelscited by 61
  7. 2015Probabilistic Numerics and Uncertainty in Computationscited by 60
  8. 2017Bayesian Probabilistic Numerical Methodscited by 51
  9. 2015On Markov chain Monte Carlo methods for tall datacited by 50
  10. 2014Sampling for Inference in Probabilistic Models with Fast Bayesian Quadraturecited by 31
  11. 2016Patterns of Scalable Bayesian Inferencecited by 25
  12. 2013Austerity in MCMC Land: Cutting the Metropolis-Hastings Budgetcited by 24
20 more
  1. 2020Variational Bayesian Monte Carlo with Noisy Likelihoodscited by 23
  2. 2018Variational Bayesian Monte Carlocited by 20
  3. 2020Generalized Posteriors in Approximate Bayesian Computationcited by 19
  4. 2019Parallel Gaussian process surrogate Bayesian inference with noisy likelihood evaluationscited by 18
  5. 2014GPS-ABC: Gaussian Process Surrogate Approximate Bayesian Computationcited by 17
  6. 2014Accelerating Asymptotically Exact MCMC for Computationally Intensive Models via Local Approximationscited by 17
  7. 2017Efficient acquisition rules for model-based approximate Bayesian computationcited by 16
  8. 2020Asymptotically Optimal Exact Minibatch Metropolis-Hastingscited by 14
  9. 2018Improving Quadrature for Constrained Integrandscited by 12
  10. 2014Noisy Monte Carlo: Convergence of Markov chains with approximate transition kernelscited by 10
  11. 2018Evaluating Gaussian Process Metamodels and Sequential Designs for Noisy Level Set Estimationcited by 10
  12. 2015Adaptive, delayed-acceptance MCMC for targets with expensive likelihoodscited by 9
  13. 2017Adaptive Gaussian process approximation for Bayesian inference with expensive likelihood functionscited by 9
  14. 2014Accelerating ABC methods using Gaussian processescited by 7
  15. 2018Robust Bayesian Synthetic Likelihood via a Semi-Parametric Approachcited by 6
  16. 2015Hamiltonian Monte Carlo Acceleration Using Surrogate Functions with Random Basescited by 5
  17. 2019Batch simulations and uncertainty quantification in Gaussian process surrogate approximate Bayesian computationcited by 5
  18. 2017Query Efficient Posterior Estimation in Scientific Experiments via Bayesian Active Learningcited by 4
  19. 2019Iterative Construction of Gaussian Process Surrogate Models for Bayesian Inferencecited by 1
  20. 2020Sequentially guided MCMC proposals for synthetic likelihoods and correlated synthetic likelihoodscited by 1

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 0; 2023: 0; 2024: 0; 2025: 2.

3 papers.

  1. 2021A survey of Monte Carlo methods for noisy and costly densities with application to reinforcement learning and ABCcited by 8
  2. 2025Measuring the right thing: justifying metrics in AI impact assessmentscited by 0
  3. 2025A Critical Review of Monte Carlo Algorithms Balancing Performance and Probabilistic Accuracy with AI Augmented Frameworkcited by 0