# Surrogate Likelihoods for Variational Annealed Importance Sampling

arXiv: https://arxiv.org/abs/2112.12194 · HTML: https://mlsanity.com/paper/2112.12194/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 68 references that are in the map.
- First posted on arXiv 22 Dec 2021.

## Builds on (most cited first, 30 of 36)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Variational Inference with Normalizing Flows](https://mlsanity.com/paper/1505.05770/) (2015)
- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Importance Weighted Autoencoders](https://mlsanity.com/paper/1509.00519/) (2015)
- [Pyro: Deep Universal Probabilistic Programming](https://mlsanity.com/paper/1810.09538/) (2018)
- [Stochastic Gradient Hamiltonian Monte Carlo](https://mlsanity.com/paper/1402.4102/) (2014)
- [Black Box Variational Inference](https://mlsanity.com/paper/1401.0118/) (2013)
- [A Conceptual Introduction to Hamiltonian Monte Carlo](https://mlsanity.com/paper/1701.02434/) (2017)
- [Composable Effects for Flexible and Accelerated Probabilistic Programming in NumPyro](https://mlsanity.com/paper/1912.11554/) (2019)
- [A Complete Recipe for Stochastic Gradient MCMC](https://mlsanity.com/paper/1506.04696/) (2015)
- [Variational Sequential Monte Carlo](https://mlsanity.com/paper/1705.11140/) (2017)
- [Filtering Variational Objectives](https://mlsanity.com/paper/1705.09279/) (2017)
- [Coresets for Scalable Bayesian Logistic Regression](https://mlsanity.com/paper/1605.06423/) (2016)
- [Practical Coreset Constructions for Machine Learning](https://mlsanity.com/paper/1703.06476/) (2017)
- [Bayesian Coreset Construction via Greedy Iterative Geodesic Ascent](https://mlsanity.com/paper/1802.01737/) (2018)
- [Auto-Encoding Sequential Monte Carlo](https://mlsanity.com/paper/1705.10306/) (2017)
- [Block Neural Autoregressive Flow](https://mlsanity.com/paper/1904.04676/) (2019)
- [Automated Scalable Bayesian Inference via Hilbert Coresets](https://mlsanity.com/paper/1710.05053/) (2017)
- [Importance Weighting and Variational Inference](https://mlsanity.com/paper/1808.09034/) (2018)
- [Hamiltonian Variational Auto-Encoder](https://mlsanity.com/paper/1805.11328/) (2018)
- [MCMC Variational Inference via Uncorrected Hamiltonian Annealing](https://mlsanity.com/paper/2107.04150/) (2021)
- [Reinterpreting Importance-Weighted Autoencoders](https://mlsanity.com/paper/1704.02916/) (2017)
- [Monte Carlo Variational Auto-Encoders](https://mlsanity.com/paper/2106.15921/) (2021)
- [Differentiable Annealed Importance Sampling and the Perils of Gradient Noise](https://mlsanity.com/paper/2107.10211/) (2021)
- [The Thermodynamic Variational Objective](https://mlsanity.com/paper/1907.00031/) (2019)
- [Fast Matrix Square Roots with Applications to Gaussian Processes and Bayesian Optimization](https://mlsanity.com/paper/2006.11267/) (2020)
- [Firefly Monte Carlo: Exact MCMC with Subsets of Data](https://mlsanity.com/paper/1403.5693/) (2014)
- [Approximate Inference with Amortised MCMC](https://mlsanity.com/paper/1702.08343/) (2017)
- [A Contrastive Divergence for Combining Variational Inference and MCMC](https://mlsanity.com/paper/1905.04062/) (2019)
- [Sandwiching the marginal likelihood using bidirectional Monte Carlo](https://mlsanity.com/paper/1511.02543/) (2015)

## Built on by (most cited first, 14 of 14)

- [Beyond ELBOs: A Large-Scale Evaluation of Variational Methods for Sampling](https://mlsanity.com/paper/2406.07423/) (2024)
- [Score-Based Diffusion meets Annealed Importance Sampling](https://mlsanity.com/paper/2208.07698/) (2022)
- [Langevin Diffusion Variational Inference](https://mlsanity.com/paper/2208.07743/) (2022)
- [Bayesian inference via sparse Hamiltonian flows](https://mlsanity.com/paper/2203.05723/) (2022)
- [MixFlows: principled variational inference via mixed flows](https://mlsanity.com/paper/2205.07475/) (2022)
- [Fast Bayesian Coresets via Subsampling and Quasi-Newton Refinement](https://mlsanity.com/paper/2203.09675/) (2022)
- [eVAE: Evolutionary Variational Autoencoder](https://mlsanity.com/paper/2301.00011/) (2023)
- [Variational Inference with Locally Enhanced Bounds for Hierarchical Models](https://mlsanity.com/paper/2203.04432/) (2022)
- [Machine Learning and the Future of Bayesian Computation](https://mlsanity.com/paper/2304.11251/) (2023)
- [Variational Learning of Gaussian Process Latent Variable Models through Stochastic Gradient Annealed Importance Sampling](https://mlsanity.com/paper/2408.06710/) (2024)
- [Reparameterized Variational Rejection Sampling](https://mlsanity.com/paper/2309.14612/) (2023)
- [General bounds on the quality of Bayesian coresets](https://mlsanity.com/paper/2405.11780/) (2024)
- [Asymptotically exact variational flows via involutive MCMC kernels](https://mlsanity.com/paper/2506.02162/) (2025)
- [MCMC-driven learning](https://mlsanity.com/paper/2402.09598/) (2024)

## Abstract (arXiv)

Variational inference is a powerful paradigm for approximate Bayesian inference with a number of appealing properties, including support for model learning and data subsampling. By contrast MCMC methods like Hamiltonian Monte Carlo do not share these properties but remain attractive since, contrary to parametric methods, MCMC is asymptotically unbiased. For these reasons researchers have sought to combine the strengths of both classes of algorithms, with recent approaches coming closer to realizing this vision in practice. However, supporting data subsampling in these hybrid methods can be a challenge, a shortcoming that we address by introducing a surrogate likelihood that can be learned jointly with other variational parameters. We argue theoretically that the resulting algorithm permits the user to make an intuitive trade-off between inference fidelity and computational cost. In an extensive empirical comparison we show that our method performs well in practice and that it is well-suited for black-box inference in probabilistic programming frameworks.