# Towards Deeper Understanding of Variational Autoencoding Models

arXiv: https://arxiv.org/abs/1702.08658 · HTML: https://mlsanity.com/paper/1702.08658/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Feb 2017.

## Built on by (most cited first, 30 of 56)

- [Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models](https://mlsanity.com/paper/2103.04922/) (2021)
- [Variational Mixture-of-Experts Autoencoders for Multi-Modal Deep Generative Models](https://mlsanity.com/paper/1911.03393/) (2019)
- [Lossy Image Compression with Conditional Diffusion Models](https://mlsanity.com/paper/2209.06950/) (2022)
- [High-Fidelity Pluralistic Image Completion with Transformers](https://mlsanity.com/paper/2103.14031/) (2021)
- [D2C: Diffusion-Denoising Models for Few-shot Conditional Generation](https://mlsanity.com/paper/2106.06819/) (2021)
- [HyperImpute: Generalized Iterative Imputation with Automatic Model Selection](https://mlsanity.com/paper/2206.07769/) (2022)
- [Diverse Image Inpainting with Bidirectional and Autoregressive Transformers](https://mlsanity.com/paper/2104.12335/) (2021)
- [Greedy Hierarchical Variational Autoencoders for Large-Scale Video Prediction](https://mlsanity.com/paper/2103.04174/) (2021)
- [ReMasker: Imputing Tabular Data with Masked Autoencoding](https://mlsanity.com/paper/2309.13793/) (2023)
- [Towards Defending against Adversarial Examples via Attack-Invariant Features](https://mlsanity.com/paper/2106.05036/) (2021)
- [Factor Analysis, Probabilistic Principal Component Analysis, Variational Inference, and Variational Autoencoder: Tutorial and Survey](https://mlsanity.com/paper/2101.00734/) (2021)
- [Creativity and Machine Learning: A Survey](https://mlsanity.com/paper/2104.02726/) (2021)
- [Precipitation Downscaling with Spatiotemporal Video Diffusion](https://mlsanity.com/paper/2312.06071/) (2023)
- [A deep generative model for probabilistic energy forecasting in power systems: normalizing flows](https://mlsanity.com/paper/2106.09370/) (2021)
- [DICE: Diverse Diffusion Model with Scoring for Trajectory Prediction](https://mlsanity.com/paper/2310.14570/) (2023)
- [Astronomia ex machina: a history, primer, and outlook on neural networks in astronomy](https://mlsanity.com/paper/2211.03796/) (2022)
- [Inference via Interpolation: Contrastive Representations Provably Enable Planning and Inference](https://mlsanity.com/paper/2403.04082/) (2024)
- [Simulating the Visual World with Artificial Intelligence: A Roadmap](https://mlsanity.com/paper/2511.08585/) (2025)
- [Hierarchical Latent Structure for Multi-Modal Vehicle Trajectory Forecasting](https://mlsanity.com/paper/2207.04624/) (2022)
- [Cauchy-Schwarz Regularized Autoencoder](https://mlsanity.com/paper/2101.02149/) (2021)
- [TopicNet: Semantic Graph-Guided Topic Discovery](https://mlsanity.com/paper/2110.14286/) (2021)
- [Compositional Learning-based Planning for Vision POMDPs](https://mlsanity.com/paper/2112.09456/) (2021)
- [An Adaptive Federated Relevance Framework for Spatial Temporal Graph Learning](https://mlsanity.com/paper/2206.03420/) (2022)
- [PAEDID: Patch Autoencoder Based Deep Image Decomposition For Pixel-level Defective Region Segmentation](https://mlsanity.com/paper/2203.14457/) (2022)
- [Hierarchical Residual Learning Based Vector Quantized Variational Autoencoder for Image Reconstruction and Generation](https://mlsanity.com/paper/2208.04554/) (2022)
- [Coupled Variational Autoencoder](https://mlsanity.com/paper/2306.02565/) (2023)
- [Leveraging Variational Autoencoders for Parameterized MMSE Estimation](https://mlsanity.com/paper/2307.05352/) (2023)
- [Controllable Distortion-Perception Tradeoff Through Latent Diffusion for Neural Image Compression](https://mlsanity.com/paper/2412.11379/) (2024)
- [Spread Flows for Manifold Modelling](https://mlsanity.com/paper/2109.14216/) (2021)
- [Deterministic training of generative autoencoders using invertible layers](https://mlsanity.com/paper/2205.09546/) (2022)

## Abstract (arXiv)

We propose a new family of optimization criteria for variational auto-encoding models, generalizing the standard evidence lower bound. We provide conditions under which they recover the data distribution and learn latent features, and formally show that common issues such as blurry samples and uninformative latent features arise when these conditions are not met. Based on these new insights, we propose a new sequential VAE model that can generate sharp samples on the LSUN image dataset based on pixel-wise reconstruction loss, and propose an optimization criterion that encourages unsupervised learning of informative latent features.