# Alpha-divergence Variational Inference Meets Importance Weighted Auto-Encoders: Methodology and Asymptotics

arXiv: https://arxiv.org/abs/2210.06226 · HTML: https://mlsanity.com/paper/2210.06226/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 33 references that are in the map.
- First posted on arXiv 12 Oct 2022.

## Builds on (most cited first, 16 of 16)

- [Variational Inference: A Review for Statisticians](https://mlsanity.com/paper/1601.00670/) (2016)
- [Importance Weighted Autoencoders](https://mlsanity.com/paper/1509.00519/) (2015)
- [R'enyi Divergence Variational Inference](https://mlsanity.com/paper/1602.02311/) (2016)
- [Filtering Variational Objectives](https://mlsanity.com/paper/1705.09279/) (2017)
- [Tighter Variational Bounds are Not Necessarily Better](https://mlsanity.com/paper/1802.04537/) (2018)
- [Importance Weighting and Variational Inference](https://mlsanity.com/paper/1808.09034/) (2018)
- [Doubly Reparameterized Gradient Estimators for Monte Carlo Objectives](https://mlsanity.com/paper/1810.04152/) (2018)
- [Challenges and Opportunities in High-dimensional Variational Inference](https://mlsanity.com/paper/2103.01085/) (2021)
- [Dropout Inference in Bayesian Neural Networks with Alpha-divergences](https://mlsanity.com/paper/1703.02914/) (2017)
- [Variational Inference with Tail-adaptive f-Divergence](https://mlsanity.com/paper/1810.11943/) (2018)
- [On the Difficulty of Unbiased Alpha Divergence Minimization](https://mlsanity.com/paper/2010.09541/) (2020)
- [Empirical Evaluation of Biased Methods for Alpha Divergence Minimization](https://mlsanity.com/paper/2105.06587/) (2021)
- [Mixture weights optimisation for Alpha-Divergence Variational Inference](https://mlsanity.com/paper/2106.05114/) (2021)
- [Monotonic Alpha-divergence Minimisation for Variational Inference](https://mlsanity.com/paper/2103.05684/) (2021)
- [Adversarial α-divergence Minimization for Bayesian Approximate Inference](https://mlsanity.com/paper/1909.06945/) (2019)
- [Meta-Learning Divergences of Variational Inference](https://mlsanity.com/paper/2007.02912/) (2020)

## Built on by (most cited first, 15 of 15)

- [Transport meets Variational Inference: Controlled Monte Carlo Diffusions](https://mlsanity.com/paper/2307.01050/) (2023)
- [A Short Survey on Importance Weighting for Machine Learning](https://mlsanity.com/paper/2403.10175/) (2024)
- [Variational Inference for Uncertainty Quantification: an Analysis of Trade-offs](https://mlsanity.com/paper/2403.13748/) (2024)
- [Variational Learning of Gaussian Process Latent Variable Models through Stochastic Gradient Annealed Importance Sampling](https://mlsanity.com/paper/2408.06710/) (2024)
- [Generalized Guarantees for Variational Inference in the Presence of Even and Elliptical Symmetry](https://mlsanity.com/paper/2511.01064/) (2025)
- [R'enyi Neural Processes](https://mlsanity.com/paper/2405.15991/) (2024)
- [Recursive Learning of Asymptotic Variational Objectives](https://mlsanity.com/paper/2411.02217/) (2024)
- [Fenchel-Young Variational Learning](https://mlsanity.com/paper/2502.10295/) (2025)
- [Spherical Cauchy Variational Autoencoders: Heavy Angular Tails and Exact KL Evaluation](https://mlsanity.com/paper/2506.21278/) (2025)
- [Symmetry Guarantees Statistic Recovery in Variational Inference](https://mlsanity.com/paper/2604.18310/) (2026)
- [Towards Model-Agnostic Posterior Approximation for Fast and Accurate Variational Autoencoders](https://mlsanity.com/paper/2403.08941/) (2024)
- [Bayesian Bandit Algorithms with Approximate Inference in Stochastic Linear Bandits](https://mlsanity.com/paper/2406.14071/) (2024)
- [Improved Variational Inference in Discrete VAEs using Error Correcting Codes](https://mlsanity.com/paper/2410.07840/) (2024)
- [Bures-Wasserstein Importance-Weighted Evidence Lower Bound: Exposition and Applications](https://mlsanity.com/paper/2602.04272/) (2026)
- [Stepwise Variational Inference with Vine Copulas](https://mlsanity.com/paper/2603.22959/) (2026)

## Abstract (arXiv)

Several algorithms involving the Variational R'enyi (VR) bound have been proposed to minimize an alpha-divergence between a target posterior distribution and a variational distribution. Despite promising empirical results, those algorithms resort to biased stochastic gradient descent procedures and thus lack theoretical guarantees. In this paper, we formalize and study the VR-IWAE bound, a generalization of the Importance Weighted Auto-Encoder (IWAE) bound. We show that the VR-IWAE bound enjoys several desirable properties and notably leads to the same stochastic gradient descent procedure as the VR bound in the reparameterized case, but this time by relying on unbiased gradient estimators. We then provide two complementary theoretical analyses of the VR-IWAE bound and thus of the standard IWAE bound. Those analyses shed light on the benefits or lack thereof of these bounds. Lastly, we illustrate our theoretical claims over toy and real-data examples.