# Optimal Preconditioning and Fisher Adaptive Langevin Sampling

arXiv: https://arxiv.org/abs/2305.14442 · HTML: https://mlsanity.com/paper/2305.14442/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 45 references that are in the map.
- First posted on arXiv 23 May 2023.

## Builds on (most cited first, 9 of 9)

- [Generalizing Hamiltonian Monte Carlo with Neural Networks](https://mlsanity.com/paper/1711.09268/) (2017)
- [A-NICE-MC: Adversarial Training for MCMC](https://mlsanity.com/paper/1706.07561/) (2017)
- [Exponential ergodicity of mirror-Langevin diffusions](https://mlsanity.com/paper/2005.09669/) (2020)
- [The Mirror Langevin Algorithm Converges with Vanishing Bias](https://mlsanity.com/paper/2109.12077/) (2021)
- [Langevin diffusions and the Metropolis-adjusted Langevin algorithm](https://mlsanity.com/paper/1309.2983/) (2013)
- [Auxiliary gradient-based sampling algorithms](https://mlsanity.com/paper/1610.09641/) (2016)
- [Ensemble preconditioning for Markov chain Monte Carlo simulation](https://mlsanity.com/paper/1607.03954/) (2016)
- [Gradient-based Adaptive Markov Chain Monte Carlo](https://mlsanity.com/paper/1911.01373/) (2019)
- [Metropolis-Hastings view on variational inference and adversarial training](https://mlsanity.com/paper/1810.07151/) (2018)

## Built on by (most cited first, 14 of 14)

- [Scalable Bayesian Learning with posteriors](https://mlsanity.com/paper/2406.00104/) (2024)
- [Equality Constrained Diffusion for Direct Trajectory Optimization](https://mlsanity.com/paper/2410.01939/) (2024)
- [Feel-Good Thompson Sampling for Contextual Bandits: a Markov Chain Monte Carlo Showdown](https://mlsanity.com/paper/2507.15290/) (2025)
- [Warped geometric information on the optimisation of Euclidean functions](https://mlsanity.com/paper/2308.08305/) (2023)
- [Divide-and-Conquer Predictive Coding: a structured Bayesian inference algorithm](https://mlsanity.com/paper/2408.05834/) (2024)
- [RODS: Robust Optimization Inspired Diffusion Sampling for Detecting and Reducing Hallucination in Generative Models](https://mlsanity.com/paper/2507.12201/) (2025)
- [A Non-asymptotic Analysis for Learning and Applying a Preconditioner in MCMC](https://mlsanity.com/paper/2602.10714/) (2026)
- [Gaussian Invariant Markov Chain Monte Carlo](https://mlsanity.com/paper/2506.21511/) (2025)
- [Subspace Langevin Monte Carlo](https://mlsanity.com/paper/2412.13928/) (2024)
- [High-dimensional Adaptive MCMC with Reduced Computational Complexity](https://mlsanity.com/paper/2604.09286/) (2026)
- [Time-Inhomogeneous Preconditioned Langevin Dynamics](https://mlsanity.com/paper/2605.06091/) (2026)
- [Event Generation with Parallel Langevin Sampling and Learned Stein Diagnostics](https://mlsanity.com/paper/2606.14854/) (2026)
- [Gradient-free Riemannian Langevin Sampler](https://mlsanity.com/paper/2607.07519/) (2026)
- [Refinement-based Flow Policy Optimization](https://mlsanity.com/paper/2609.15123/) (2026)

## Abstract (arXiv)

We define an optimal preconditioning for the Langevin diffusion by analytically optimizing the expected squared jumped distance. This yields as the optimal preconditioning an inverse Fisher information covariance matrix, where the covariance matrix is computed as the outer product of log target gradients averaged under the target. We apply this result to the Metropolis adjusted Langevin algorithm (MALA) and derive a computationally efficient adaptive MCMC scheme that learns the preconditioning from the history of gradients produced as the algorithm runs. We show in several experiments that the proposed algorithm is very robust in high dimensions and significantly outperforms other methods, including a closely related adaptive MALA scheme that learns the preconditioning with standard adaptive MCMC as well as the position-dependent Riemannian manifold MALA sampler.