# Reinterpreting Importance-Weighted Autoencoders

arXiv: https://arxiv.org/abs/1704.02916 · HTML: https://mlsanity.com/paper/1704.02916/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Apr 2017.

## Built on by (most cited first, 30 of 37)

- [Variational Mixture-of-Experts Autoencoders for Multi-Modal Deep Generative Models](https://mlsanity.com/paper/1911.03393/) (2019)
- [Reasoning to Learn from Latent Thoughts](https://mlsanity.com/paper/2503.18866/) (2025)
- [Variational methods for simulation-based inference](https://mlsanity.com/paper/2203.04176/) (2022)
- [MCMC Variational Inference via Uncorrected Hamiltonian Annealing](https://mlsanity.com/paper/2107.04150/) (2021)
- [Decomposed Mutual Information Estimation for Contrastive Representation Learning](https://mlsanity.com/paper/2106.13401/) (2021)
- [Improving Lossless Compression Rates via Monte Carlo Bits-Back Coding](https://mlsanity.com/paper/2102.11086/) (2021)
- [Surrogate Likelihoods for Variational Annealed Importance Sampling](https://mlsanity.com/paper/2112.12194/) (2021)
- [A likelihood approach to nonparametric estimation of a singular distribution using deep generative models](https://mlsanity.com/paper/2105.04046/) (2021)
- [Recursive Monte Carlo and Variational Inference with Auxiliary Variables](https://mlsanity.com/paper/2203.02836/) (2022)
- [Variational Sparse Coding with Learned Thresholding](https://mlsanity.com/paper/2205.03665/) (2022)
- [Scalable Multi-Task Gaussian Processes with Neural Embedding of Coregionalization](https://mlsanity.com/paper/2109.09261/) (2021)
- [Multi-Sample Training for Neural Image Compression](https://mlsanity.com/paper/2209.13834/) (2022)
- [Variational Inference with Locally Enhanced Bounds for Hierarchical Models](https://mlsanity.com/paper/2203.04432/) (2022)
- [Black-box Coreset Variational Inference](https://mlsanity.com/paper/2211.02377/) (2022)
- [An Easy to Interpret Diagnostic for Approximate Inference: Symmetric Divergence Over Simulations](https://mlsanity.com/paper/2103.01030/) (2021)
- [Deep Generative Pattern-Set Mixture Models for Nonignorable Missingness](https://mlsanity.com/paper/2103.03532/) (2021)
- [U-Statistics for Importance-Weighted Variational Inference](https://mlsanity.com/paper/2302.13918/) (2023)
- [Self-Supervised Bootstrapping of Action-Predictive Embodied Reasoning](https://mlsanity.com/paper/2602.08167/) (2026)
- [Deeply-Learned Generalized Linear Models with Missing Data](https://mlsanity.com/paper/2207.08911/) (2022)
- [Unscented Autoencoder](https://mlsanity.com/paper/2306.05256/) (2023)
- [Reparameterized Variational Rejection Sampling](https://mlsanity.com/paper/2309.14612/) (2023)
- [Improving Variational Autoencoder Estimation from Incomplete Data with Mixture Variational Families](https://mlsanity.com/paper/2403.03069/) (2024)
- [Differentiable Annealed Importance Sampling Minimizes The Symmetrized Kullback-Leibler Divergence Between Initial and Target Distribution](https://mlsanity.com/paper/2405.14840/) (2024)
- [Unsupervised Imputation of Non-ignorably Missing Data Using Importance-Weighted Autoencoders](https://mlsanity.com/paper/2101.07357/) (2021)
- [NEO: Non Equilibrium Sampling on the Orbit of a Deterministic Transform](https://mlsanity.com/paper/2103.10943/) (2021)
- [Latent Variable Modelling Using Variational Autoencoders: A survey](https://mlsanity.com/paper/2206.09891/) (2022)
- [Decomposed Mutual Information Optimization for Generalized Context in Meta-Reinforcement Learning](https://mlsanity.com/paper/2210.04209/) (2022)
- [Massively Parallel Reweighted Wake-Sleep](https://mlsanity.com/paper/2305.11022/) (2023)
- [Deep Probabilistic Graphical Modeling](https://mlsanity.com/paper/2104.12053/) (2021)
- [Tighter Variational Bounds are Not Necessarily Better. A Research Report on Implementation, Ablation Study, and Extensions](https://mlsanity.com/paper/2209.11875/) (2022)

## Abstract (arXiv)

The standard interpretation of importance-weighted autoencoders is that they maximize a tighter lower bound on the marginal likelihood than the standard evidence lower bound. We give an alternate interpretation of this procedure: that it optimizes the standard variational lower bound, but using a more complex distribution. We formally derive this result, present a tighter lower bound, and visualize the implicit importance-weighted distribution.