mlsanityData to 7 Oct 2026

Reinforcement Learning for Adaptive MCMC

arXiv 2405.13574 ↗First posted 22 May 2024stat.CO cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

An informal observation, made by several authors, is that the adaptive design of a Markov transition kernel has the flavour of a reinforcement learning task. Yet, to-date it has remained unclear how to actually exploit modern reinforcement learning technologies for adaptive MCMC. The aim of this paper is to set out a general framework, called Reinforcement Learning Metropolis--Hastings, that is theoretically supported and empirically validated. Our principal focus is on learning fast-mixing Metropolis--Hastings transition kernels, which we cast as deterministic policies and optimise via a policy gradient. Control of the learning rate provably ensures conditions for ergodicity are satisfied. The methodology is used to construct a gradient-free sampler that out-performs a popular gradient-free adaptive Metropolis--Hastings algorithm on ≈ 90 % of tasks in the PosteriorDB benchmark.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 15 of its 55 references that are in the map. First posted on arXiv 22 May 2024.

Builds on

15 of 55 references.

References · 15

  1. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  2. 2015Continuous control with deep reinforcement learningcited by 3,279
  3. 2020Deep Reinforcement Learning for Autonomous Driving: A Surveycited by 698
  4. 2018An Introduction to Deep Reinforcement Learningcited by 201
  5. 2017Parameter Space Noise for Explorationcited by 158
  6. 2022Exploration in Deep Reinforcement Learning: A Surveycited by 158
  7. 2019The Barker proposal: combining robustness and efficiency in gradient-based MCMCcited by 20
  8. 2014Accelerating Asymptotically Exact MCMC for Computationally Intensive Models via Local Approximationscited by 17
  9. 2019Gradient-based Adaptive Markov Chain Monte Carlocited by 14
  10. 2021Efficient and Generalizable Tuning Strategies for Stochastic Gradient MCMCcited by 11
  11. 2023Accelerating Markov Chain Monte Carlo sampling with diffusion modelscited by 10
  12. 2023Self-Tuning Hamiltonian Monte Carlo for Accelerated Samplingcited by 4
3 more
  1. 2020Multi-agent Reinforcement Learning Accelerated MCMC on Multiscale Inversion Problemcited by 3
  2. 2013Adaptive Metropolis-Hastings Sampling using Reversible Dependent Mixture Proposalscited by 2
  3. 2023Policy Gradient Optimal Correlation Search for Variance Reduction in Monte Carlo simulation and Maximum Optimal Transportcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 1; 2026: 3.

4 papers.

  1. 2025Harnessing the Power of Reinforcement Learning for Adaptive MCMCcited by 2
  2. 2026Neural Predictor-Corrector: Solving Homotopy Problems with Reinforcement Learningcited by 2
  3. 2026Stop the Sampler! Classifier-Based Adaptive Stopping for Sampling Kernelscited by 0
  4. 2026GRPO-QPS: Target-Preserving Reinforcement Learning for Quantum Posterior Samplingcited by 0