# Ensemble-Based Annealed Importance Sampling

arXiv: https://arxiv.org/abs/2401.15645 · HTML: https://mlsanity.com/paper/2401.15645/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 103 references that are in the map.
- First posted on arXiv 28 Jan 2024.

## Builds on (most cited first, 30 of 34)

- [Stochastic Interpolants: A Unifying Framework for Flows and Diffusions](https://mlsanity.com/paper/2303.08797/) (2023)
- [Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm](https://mlsanity.com/paper/1608.04471/) (2016)
- [Theoretical guarantees for approximate sampling from smooth and log-concave densities](https://mlsanity.com/paper/1412.7392/) (2014)
- [Non-asymptotic convergence analysis for the Unadjusted Langevin Algorithm](https://mlsanity.com/paper/1507.05021/) (2015)
- [Stein Variational Gradient Descent as Gradient Flow](https://mlsanity.com/paper/1704.07520/) (2017)
- [Denoising Diffusion Samplers](https://mlsanity.com/paper/2302.13834/) (2023)
- [Underdamped Langevin MCMC: A non-asymptotic analysis](https://mlsanity.com/paper/1707.03663/) (2017)
- [Flow Annealed Importance Sampling Bootstrap](https://mlsanity.com/paper/2208.01893/) (2022)
- [Log-concave sampling: Metropolis-Hastings algorithms are fast](https://mlsanity.com/paper/1801.02309/) (2018)
- [Score-Based Diffusion meets Annealed Importance Sampling](https://mlsanity.com/paper/2208.07698/) (2022)
- [Accelerating Langevin Sampling with Birth-death](https://mlsanity.com/paper/1905.09863/) (2019)
- [Minimax Mixing Time of the Metropolis-Adjusted Langevin Algorithm for Log-Concave Sampling](https://mlsanity.com/paper/2109.13055/) (2021)
- [Accelerated Information Gradient flow](https://mlsanity.com/paper/1909.02102/) (2019)
- [Differentiable Annealed Importance Sampling and the Perils of Gradient Noise](https://mlsanity.com/paper/2107.10211/) (2021)
- [Scaling limit of the Stein variational gradient descent: the mean field regime](https://mlsanity.com/paper/1805.04035/) (2018)
- [Birth-death dynamics for sampling: Global convergence, approximations and their asymptotics](https://mlsanity.com/paper/2211.00450/) (2022)
- [The Bouncy Particle Sampler: A Non-Reversible Rejection-Free Markov Chain Monte Carlo Method](https://mlsanity.com/paper/1510.02451/) (2015)
- [Sampling in Unit Time with Kernel Fisher-Rao Flow](https://mlsanity.com/paper/2401.03892/) (2024)
- [Sampling via Gradient Flows in the Space of Probability Measures](https://mlsanity.com/paper/2310.03597/) (2023)
- [On the Geometric Ergodicity of Hamiltonian Monte Carlo](https://mlsanity.com/paper/1601.08057/) (2016)
- [Ensemble preconditioning for Markov chain Monte Carlo simulation](https://mlsanity.com/paper/1607.03954/) (2016)
- [Efficient Training of Energy-Based Models Using Jarzynski Equality](https://mlsanity.com/paper/2305.19414/) (2023)
- [Rapid Mixing of Hamiltonian Monte Carlo on Strongly Log-Concave Distributions](https://mlsanity.com/paper/1708.07114/) (2017)
- [Learning Deep Generative Models with Annealed Importance Sampling](https://mlsanity.com/paper/1906.04904/) (2019)
- [Generative Modeling via Hierarchical Tensor Sketching](https://mlsanity.com/paper/2304.05305/) (2023)
- [Second Order Ensemble Langevin Method for Sampling and Inverse Problems](https://mlsanity.com/paper/2208.04506/) (2022)
- [Complexity of zigzag sampling algorithm for strongly log-concave distributions](https://mlsanity.com/paper/2012.11094/) (2020)
- [A gradient-free subspace-adjusting ensemble sampler for infinite-dimensional Bayesian inverse problems](https://mlsanity.com/paper/2202.11088/) (2022)
- [Free Energy Evaluation Using Marginalized Annealed Importance Sampling](https://mlsanity.com/paper/2204.03784/) (2022)
- [Learning Optimal Flows for Non-Equilibrium Importance Sampling](https://mlsanity.com/paper/2206.09908/) (2022)

## Built on by (most cited first, 4 of 4)

- [Solving Inverse Problems via Diffusion-Based Priors: An Approximation-Free Ensemble Sampling Approach](https://mlsanity.com/paper/2506.03979/) (2025)
- [Complexity Analysis of Normalizing Constant Estimation: from Jarzynski Equality to Annealed Importance Sampling and beyond](https://mlsanity.com/paper/2502.04575/) (2025)
- [Efficient, Multimodal, and Derivative-Free Bayesian Inference With Fisher-Rao Gradient Flows](https://mlsanity.com/paper/2406.17263/) (2024)
- [New affine invariant ensemble samplers and their dimensional scaling](https://mlsanity.com/paper/2505.02987/) (2025)

## Abstract (arXiv)

Sampling from a multimodal distribution is a fundamental and challenging problem in computational science and statistics. Among various approaches proposed for this task, one popular method is Annealed Importance Sampling (AIS). In this paper, we propose an ensemble-based version of AIS by combining it with population-based Monte Carlo methods to improve its efficiency. By keeping track of an ensemble instead of a single particle along some continuation path between the starting distribution and the target distribution, we take advantage of the interaction within the ensemble to encourage the exploration of undiscovered modes. Specifically, our main idea is to utilize either the snooker algorithm or the genetic algorithm used in Evolutionary Monte Carlo. We discuss how the proposed algorithm can be implemented and derive a partial differential equation governing the evolution of the ensemble under the continuous time and mean-field limit. We also test the efficiency of the proposed algorithm on various continuous and discrete distributions.