mlsanityData to 7 Oct 2026

A survey of Monte Carlo methods for noisy and costly densities with application to reinforcement learning and ABC

arXiv 2108.00490 ↗First posted 1 Aug 2021cs.LG stat.CO stat.MEcited by 8

Not on the map. Why?

Abstract (arXiv)

This survey gives an overview of Monte Carlo methodologies using surrogate models, for dealing with densities which are intractable, costly, and/or noisy. This type of problem can be found in numerous real-world scenarios, including stochastic optimization and reinforcement learning, where each evaluation of a density function may incur some computationally-expensive or even physical (real-world activity) cost, likely to give different results each time. The surrogate model does not incur this cost, but there are important trade-offs and considerations involved in the choice and design of such methodologies. We classify the different methodologies into three main classes and describe specific instances of algorithms under a unified notation. A modular scheme which encompasses the considered methods is also presented. A range of application scenarios is discussed, with special attention to the likelihood-free setting and reinforcement learning. Several numerical comparisons are also provided.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Builds on 33 of its 101 references that are in the map. First posted on arXiv 1 Aug 2021.

Builds on

33 of 101 references.

References · 33

  1. 2015Bayesian Optimization for Likelihood-Free Inference of Simulator-Based Statistical Modelscited by 61
  2. 2013Local Gaussian process approximation for large computer experimentscited by 59
  3. 2015On Markov chain Monte Carlo methods for tall datacited by 50
  4. 2018A survey on policy search algorithms for learning robot controllers in a handful of trialscited by 33
  5. 2020Marginal likelihood computation for model selection and hypothesis testing: an extensive reviewcited by 33
  6. 2015Probabilistic Integration: A Role in Statistical Computation?cited by 26
  7. 2020Variational Bayesian Monte Carlo with Noisy Likelihoodscited by 23
  8. 2021A Survey of Monte Carlo Methods for Parameter Estimationcited by 23
  9. 2019Convergence Guarantees for Adaptive Bayesian Quadrature Methodscited by 22
  10. 2014Speeding Up MCMC by Efficient Data Subsamplingcited by 18
  11. 2019Parallel Gaussian process surrogate Bayesian inference with noisy likelihood evaluationscited by 18
  12. 2014GPS-ABC: Gaussian Process Surrogate Approximate Bayesian Computationcited by 17
21 more
  1. 2014Accelerating Asymptotically Exact MCMC for Computationally Intensive Models via Local Approximationscited by 17
  2. 2017Efficient acquisition rules for model-based approximate Bayesian computationcited by 16
  3. 2019Active emulation of computer codes with Gaussian processes -- Application to remote sensingcited by 11
  4. 2014Noisy Monte Carlo: Convergence of Markov chains with approximate transition kernelscited by 10
  5. 2018A Bayes-Sard Cubature Methodcited by 10
  6. 2020A Joint introduction to Gaussian Processes and Relevance Vector Machines with Connections to Kalman filtering and other Kernel Smootherscited by 10
  7. 2015Adaptive, delayed-acceptance MCMC for targets with expensive likelihoodscited by 9
  8. 2017Adaptive Gaussian process approximation for Bayesian inference with expensive likelihood functionscited by 9
  9. 2020Deep Importance Sampling based on Regression for Model Inversion and Emulationcited by 8
  10. 2021Ensemble Inference Methods for Models With Noisy and Expensive Likelihoodscited by 8
  11. 2014Accelerating ABC methods using Gaussian processescited by 7
  12. 2020Adaptive quadrature schemes for Bayesian inference via active learningcited by 6
  13. 2016The Recycling Gibbs Sampler for Efficient Learningcited by 5
  14. 2013Adaptive Independent Sticky MCMC algorithmscited by 4
  15. 2017Query Efficient Posterior Estimation in Scientific Experiments via Bayesian Active Learningcited by 4
  16. 2015Speeding Up MCMC by Delayed Acceptance and Data Subsamplingcited by 3
  17. 2021Approximate Bayesian inference from noisy likelihoods with Gaussian process emulated MCMCcited by 3
  18. 2013Importance sampling squared for Bayesian inference in latent variable modelscited by 2
  19. 2016Parallel local approximation MCMC for expensive modelscited by 2
  20. 2020Accelerating sequential Monte Carlo with surrogate likelihoodscited by 2
  21. 2020Moving Target Monte Carlocited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 2; 2024: 0; 2025: 0; 2026: 3.

8 papers.

  1. 2022ISFL: Federated Learning for Non-i.i.d. Data with Local Importance Samplingcited by 4
  2. 2022Optimality in Noisy Importance Samplingcited by 2
  3. 2026The Normalized Maximum Likelihood for Regular Non-Smooth Models: Measure-Theoretic Foundations and Geometric Samplingcited by 1
  4. 2022CAMEO: Curiosity Augmented Metropolis for Exploratory Optimal Policiescited by 0
  5. 2023Deep Metric Tensor Regularized Policy Gradientcited by 0
  6. 2023Binary classification based Monte Carlo simulationcited by 0
  7. 2026A unifying view of contrastive learning, importance sampling, and bridge sampling for energy-based modelscited by 0
  8. 2026AME: A Multi-Type Contributor Attribution Framework in Generative AI Marketscited by 0