# Optimization of Annealed Importance Sampling Hyperparameters

arXiv: https://arxiv.org/abs/2209.13226 · HTML: https://mlsanity.com/paper/2209.13226/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 52 references that are in the map.
- First posted on arXiv 27 Sep 2022.

## Builds on (most cited first, 30 of 35)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Improving Variational Inference with Inverse Autoregressive Flow](https://mlsanity.com/paper/1606.04934/) (2016)
- [Importance Weighted Autoencoders](https://mlsanity.com/paper/1509.00519/) (2015)
- [Assessing Generative Models via Precision and Recall](https://mlsanity.com/paper/1806.00035/) (2018)
- [Adversarial Feature Learning](https://mlsanity.com/paper/1605.09782/) (2016)
- [Black Box Variational Inference](https://mlsanity.com/paper/1401.0118/) (2013)
- [A note on the evaluation of generative models](https://mlsanity.com/paper/1511.01844/) (2015)
- [Stochastic Normalizing Flows](https://mlsanity.com/paper/2002.06707/) (2020)
- [Variational Sequential Monte Carlo](https://mlsanity.com/paper/1705.11140/) (2017)
- [Filtering Variational Objectives](https://mlsanity.com/paper/1705.09279/) (2017)
- [Annealed Flow Transport Monte Carlo](https://mlsanity.com/paper/2102.07501/) (2021)
- [Variational inference for Monte Carlo objectives](https://mlsanity.com/paper/1602.06725/) (2016)
- [Auto-Encoding Sequential Monte Carlo](https://mlsanity.com/paper/1705.10306/) (2017)
- [Importance Weighting and Variational Inference](https://mlsanity.com/paper/1808.09034/) (2018)
- [Hamiltonian Variational Auto-Encoder](https://mlsanity.com/paper/1805.11328/) (2018)
- [On the Quantitative Analysis of Decoder-Based Generative Models](https://mlsanity.com/paper/1611.04273/) (2016)
- [MCMC Variational Inference via Uncorrected Hamiltonian Annealing](https://mlsanity.com/paper/2107.04150/) (2021)
- [Monte Carlo Variational Auto-Encoders](https://mlsanity.com/paper/2106.15921/) (2021)
- [Differentiable Annealed Importance Sampling and the Perils of Gradient Noise](https://mlsanity.com/paper/2107.10211/) (2021)
- [Neural Adaptive Sequential Monte Carlo](https://mlsanity.com/paper/1506.03338/) (2015)
- [Reparameterization Gradients through Acceptance-Rejection Sampling Algorithms](https://mlsanity.com/paper/1610.05683/) (2016)
- [Sandwiching the marginal likelihood using bidirectional Monte Carlo](https://mlsanity.com/paper/1511.02543/) (2015)
- [Nested Sequential Monte Carlo Methods](https://mlsanity.com/paper/1502.02536/) (2015)
- [Variational Inference with Hamiltonian Monte Carlo](https://mlsanity.com/paper/1609.08203/) (2016)
- [Evaluating Lossy Compression Rates of Deep Generative Models](https://mlsanity.com/paper/2008.06653/) (2020)

## Built on by (most cited first, 1 of 1)

- [Adaptive Annealed Importance Sampling with Constant Rate Progress](https://mlsanity.com/paper/2306.15283/) (2023)

## Abstract (arXiv)

Annealed Importance Sampling (AIS) is a popular algorithm used to estimates the intractable marginal likelihood of deep generative models. Although AIS is guaranteed to provide unbiased estimate for any set of hyperparameters, the common implementations rely on simple heuristics such as the geometric average bridging distributions between initial and the target distribution which affect the estimation performance when the computation budget is limited. In order to reduce the number of sampling iterations, we present a parameteric AIS process with flexible intermediary distributions defined by a residual density with respect to the geometric mean path. Our method allows parameter sharing between annealing distributions, the use of fix linear schedule for discretization and amortization of hyperparameter selection in latent variable models. We assess the performance of Optimized-Path AIS for marginal likelihood estimation of deep generative models and compare it to compare it to more computationally intensive AIS.