# Gradient-based Adaptive Markov Chain Monte Carlo

arXiv: https://arxiv.org/abs/1911.01373 · HTML: https://mlsanity.com/paper/1911.01373/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Nov 2019.

## Built on by (most cited first, 14 of 14)

- [Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models](https://mlsanity.com/paper/2103.04922/) (2021)
- [Path Integral Sampler: a stochastic control approach for sampling](https://mlsanity.com/paper/2111.15141/) (2021)
- [Timewarp: Transferable Acceleration of Molecular Dynamics by Learning Time-Coarsened Dynamics](https://mlsanity.com/paper/2302.01170/) (2023)
- [Optimal Preconditioning and Fisher Adaptive Langevin Sampling](https://mlsanity.com/paper/2305.14442/) (2023)
- [Optimal Scaling for Locally Balanced Proposals in Discrete Spaces](https://mlsanity.com/paper/2209.08183/) (2022)
- [Reinforcement Learning for Adaptive MCMC](https://mlsanity.com/paper/2405.13574/) (2024)
- [Entropy-based adaptive Hamiltonian Monte Carlo](https://mlsanity.com/paper/2110.14625/) (2021)
- [Score-Based Metropolis-Hastings Algorithms](https://mlsanity.com/paper/2501.00467/) (2024)
- [Harnessing the Power of Reinforcement Learning for Adaptive MCMC](https://mlsanity.com/paper/2507.00671/) (2025)
- [Semi-Empirical Objective Functions for MCMC Proposal Optimization](https://mlsanity.com/paper/2106.02104/) (2021)
- [Ai-Sampler: Adversarial Learning of Markov kernels with involutive maps](https://mlsanity.com/paper/2406.02490/) (2024)
- [Gaussian Invariant Markov Chain Monte Carlo](https://mlsanity.com/paper/2506.21511/) (2025)
- [Learning variational autoencoders via MCMC speed measures](https://mlsanity.com/paper/2308.13731/) (2023)
- [Stop the Sampler! Classifier-Based Adaptive Stopping for Sampling Kernels](https://mlsanity.com/paper/2606.16073/) (2026)

## Abstract (arXiv)

We introduce a gradient-based learning method to automatically adapt Markov chain Monte Carlo (MCMC) proposal distributions to intractable targets. We define a maximum entropy regularised objective function, referred to as generalised speed measure, which can be robustly optimised over the parameters of the proposal distribution by applying stochastic gradient optimisation. An advantage of our method compared to traditional adaptive MCMC methods is that the adaptation occurs even when candidate state values are rejected. This is a highly desirable property of any adaptation strategy because the adaptation starts in early iterations even if the initial proposal distribution is far from optimum. We apply the framework for learning multivariate random walk Metropolis and Metropolis-adjusted Langevin proposals with full covariance matrices, and provide empirical evidence that our method can outperform other MCMC algorithms, including Hamiltonian Monte Carlo schemes.