# A Framework for Adaptive MCMC Targeting Multimodal Distributions

arXiv: https://arxiv.org/abs/1812.02609 · HTML: https://mlsanity.com/paper/1812.02609/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Dec 2018.

## Built on by (most cited first, 21 of 21)

- [GFlowNet Foundations](https://mlsanity.com/paper/2111.09266/) (2021)
- [Local-Global MCMC kernels: the best of both worlds](https://mlsanity.com/paper/2111.02702/) (2021)
- [Diffusive Gibbs Sampling](https://mlsanity.com/paper/2402.03008/) (2024)
- [Training Neural Samplers with Reverse Diffusive KL Divergence](https://mlsanity.com/paper/2410.12456/) (2024)
- [A Theory of Non-Acyclic Generative Flow Networks](https://mlsanity.com/paper/2312.15246/) (2023)
- [Kernel Stein Discrepancy thinning: a theoretical perspective of pathologies and a practical fix with regularization](https://mlsanity.com/paper/2301.13528/) (2023)
- [Learned Reference-based Diffusion Sampling for multi-modal distributions](https://mlsanity.com/paper/2410.19449/) (2024)
- [LSB: Local Self-Balancing MCMC in Discrete Spaces](https://mlsanity.com/paper/2109.03867/) (2021)
- [Energy-Weighted Flow Matching: Unlocking Continuous Normalizing Flows for Efficient and Scalable Boltzmann Sampling](https://mlsanity.com/paper/2509.03726/) (2025)
- [Stereographic Markov Chain Monte Carlo](https://mlsanity.com/paper/2205.12112/) (2022)
- [Machine Learning and the Future of Bayesian Computation](https://mlsanity.com/paper/2304.11251/) (2023)
- [Using Perturbation to Improve Goodness-of-Fit Tests based on Kernelized Stein Discrepancy](https://mlsanity.com/paper/2304.14762/) (2023)
- [Reflection coupling for unadjusted generalized Hamiltonian Monte Carlo in the nonconvex stochastic gradient case](https://mlsanity.com/paper/2310.18774/) (2023)
- [On a Neural Implementation of Brenier's Polar Factorization](https://mlsanity.com/paper/2403.03071/) (2024)
- [Diverse Policy Optimization for Structured Action Space](https://mlsanity.com/paper/2302.11917/) (2023)
- [Symmetry-driven embedding of networks in hyperbolic space](https://mlsanity.com/paper/2406.10711/) (2024)
- [Slithering Through Gaps: Capturing Discrete Isolated Modes via Logistic Bridging](https://mlsanity.com/paper/2604.10821/) (2026)
- [Cyclical Kernel Adaptive Metropolis](https://mlsanity.com/paper/2206.14421/) (2022)
- [Repelling-Attracting Hamiltonian Monte Carlo](https://mlsanity.com/paper/2403.04607/) (2024)
- [Branching Stein Variational Gradient Descent for sampling multimodal distributions](https://mlsanity.com/paper/2506.13916/) (2025)
- [Discrete Causal Representations from Heterogeneous Domains: A Bayesian Approach with Social Survey Applications](https://mlsanity.com/paper/2606.06288/) (2026)

## Abstract (arXiv)

We propose a new Monte Carlo method for sampling from multimodal distributions. The idea of this technique is based on splitting the task into two: finding the modes of a target distribution π and sampling, given the knowledge of the locations of the modes. The sampling algorithm relies on steps of two types: local ones, preserving the mode; and jumps to regions associated with different modes. Besides, the method learns the optimal parameters of the algorithm while it runs, without requiring user intervention. Our technique should be considered as a flexible framework, in which the design of moves can follow various strategies known from the broad MCMC literature. In order to design an adaptive scheme that facilitates both local and jump moves, we introduce an auxiliary variable representing each mode and we define a new target distribution tildeπ on an augmented state space X~×~I, where X is the original state space of π and I is the set of the modes. As the algorithm runs and updates its parameters, the target distribution tildeπ also keeps being modified. This motivates a new class of algorithms, Auxiliary Variable Adaptive MCMC. We prove general ergodic results for the whole class before specialising to the case of our algorithm.