# Adaptive Annealed Importance Sampling with Constant Rate Progress

arXiv: https://arxiv.org/abs/2306.15283 · HTML: https://mlsanity.com/paper/2306.15283/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 41 references that are in the map.
- First posted on arXiv 27 Jun 2023.

## Builds on (most cited first, 26 of 26)

- [Stein Variational Gradient Descent: A General Purpose Bayesian Inference Algorithm](https://mlsanity.com/paper/1608.04471/) (2016)
- [Importance Weighted Autoencoders](https://mlsanity.com/paper/1509.00519/) (2015)
- [Stochastic Normalizing Flows](https://mlsanity.com/paper/2002.06707/) (2020)
- [Stein Variational Gradient Descent as Gradient Flow](https://mlsanity.com/paper/1704.07520/) (2017)
- [Variational Sequential Monte Carlo](https://mlsanity.com/paper/1705.11140/) (2017)
- [Filtering Variational Objectives](https://mlsanity.com/paper/1705.09279/) (2017)
- [Annealed Flow Transport Monte Carlo](https://mlsanity.com/paper/2102.07501/) (2021)
- [Importance Weighting and Variational Inference](https://mlsanity.com/paper/1808.09034/) (2018)
- [Score-Based Diffusion meets Annealed Importance Sampling](https://mlsanity.com/paper/2208.07698/) (2022)
- [On the Quantitative Analysis of Decoder-Based Generative Models](https://mlsanity.com/paper/1611.04273/) (2016)
- [MCMC Variational Inference via Uncorrected Hamiltonian Annealing](https://mlsanity.com/paper/2107.04150/) (2021)
- [Monte Carlo Variational Auto-Encoders](https://mlsanity.com/paper/2106.15921/) (2021)
- [Differentiable Annealed Importance Sampling and the Perils of Gradient Noise](https://mlsanity.com/paper/2107.10211/) (2021)
- [The Thermodynamic Variational Objective](https://mlsanity.com/paper/1907.00031/) (2019)
- [Annealed Stein Variational Gradient Descent](https://mlsanity.com/paper/2101.09815/) (2021)
- [Provable Bayesian Inference via Particle Mirror Descent](https://mlsanity.com/paper/1506.03101/) (2015)
- [Sandwiching the marginal likelihood using bidirectional Monte Carlo](https://mlsanity.com/paper/1511.02543/) (2015)
- [Wasserstein Proximal of GANs](https://mlsanity.com/paper/2102.06862/) (2021)
- [Approximate inference with Wasserstein gradient flows](https://mlsanity.com/paper/1806.04542/) (2018)
- [q-Paths: Generalizing the Geometric Annealing Path using Power Means](https://mlsanity.com/paper/2107.00745/) (2021)
- [Probability Functional Descent: A Unifying Perspective on GANs, Variational Inference, and Reinforcement Learning](https://mlsanity.com/paper/1901.10691/) (2019)
- [Measuring the reliability of MCMC inference with bidirectional Monte Carlo](https://mlsanity.com/paper/1606.02275/) (2016)
- [Annealed Importance Sampling with q-Paths](https://mlsanity.com/paper/2012.07823/) (2020)
- [Variational Optimization of Annealing Schedules](https://mlsanity.com/paper/1502.05313/) (2015)
- [Towards Automatic Model Comparison: An Adaptive Sequential Monte Carlo Approach](https://mlsanity.com/paper/1303.3123/) (2013)
- [Optimization of Annealed Importance Sampling Hyperparameters](https://mlsanity.com/paper/2209.13226/) (2022)

## Built on by (most cited first, 7 of 7)

- [Sampling in Unit Time with Kernel Fisher-Rao Flow](https://mlsanity.com/paper/2401.03892/) (2024)
- [Transport meets Variational Inference: Controlled Monte Carlo Diffusions](https://mlsanity.com/paper/2307.01050/) (2023)
- [A connection between Tempering and Entropic Mirror Descent](https://mlsanity.com/paper/2310.11914/) (2023)
- [Temperature-Annealed Boltzmann Generators](https://mlsanity.com/paper/2501.19077/) (2025)
- [Provable benefits of annealing for estimating normalizing constants: Importance Sampling, Noise-Contrastive Estimation, and beyond](https://mlsanity.com/paper/2310.03902/) (2023)
- [Learning Boltzmann Generators via Constrained Mass Transport](https://mlsanity.com/paper/2510.18460/) (2025)
- [Properties and limitations of geometric tempering for gradient flow dynamics](https://mlsanity.com/paper/2604.20301/) (2026)

## Abstract (arXiv)

Annealed Importance Sampling (AIS) synthesizes weighted samples from an intractable distribution given its unnormalized density function. This algorithm relies on a sequence of interpolating distributions bridging the target to an initial tractable distribution such as the well-known geometric mean path of unnormalized distributions which is assumed to be suboptimal in general. In this paper, we prove that the geometric annealing corresponds to the distribution path that minimizes the KL divergence between the current particle distribution and the desired target when the feasible change in the particle distribution is constrained. Following this observation, we derive the constant rate discretization schedule for this annealing sequence, which adjusts the schedule to the difficulty of moving samples between the initial and the target distributions. We further extend our results to f-divergences and present the respective dynamics of annealing sequences based on which we propose the Constant Rate AIS (CR-AIS) algorithm and its efficient implementation for α-divergences. We empirically show that CR-AIS performs well on multiple benchmark distributions while avoiding the computationally expensive tuning loop in existing Adaptive AIS.