# The Usual Suspects? Reassessing Blame for VAE Posterior Collapse

arXiv: https://arxiv.org/abs/1912.10702 · HTML: https://mlsanity.com/paper/1912.10702/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Dec 2019.

## Built on by (most cited first, 30 of 52)

- [Identifiability of deep generative models without auxiliary information](https://mlsanity.com/paper/2206.10044/) (2022)
- [Posterior Collapse and Latent Variable Non-identifiability](https://mlsanity.com/paper/2301.00537/) (2023)
- [A Critical Look at the Consistency of Causal Estimation With Deep Latent Variable Models](https://mlsanity.com/paper/2102.06648/) (2021)
- [Embrace the Gap: VAEs Perform Independent Mechanism Analysis](https://mlsanity.com/paper/2206.02416/) (2022)
- [Posterior Collapse of a Linear Latent Variable Model](https://mlsanity.com/paper/2205.04009/) (2022)
- [Multi-Rate VAE: Train Once, Get the Full Rate-Distortion Curve](https://mlsanity.com/paper/2212.03905/) (2022)
- [Preventing Oversmoothing in VAE via Generalized Variance Parameterization](https://mlsanity.com/paper/2102.08663/) (2021)
- [Flow Matching Meets Biology and Life Science: A Survey](https://mlsanity.com/paper/2507.17731/) (2025)
- [VAE Approximation Error: ELBO and Exponential Families](https://mlsanity.com/paper/2102.09310/) (2021)
- [Variational autoencoders in the presence of low-dimensional data: landscape and implicit bias](https://mlsanity.com/paper/2112.06868/) (2021)
- [How do Variational Autoencoders Learn? Insights from Representational Similarity](https://mlsanity.com/paper/2205.08399/) (2022)
- [Camera-Conditioned Stable Feature Generation for Isolated Camera Supervised Person Re-IDentification](https://mlsanity.com/paper/2203.15210/) (2022)
- [Be More Active! Understanding the Differences between Mean and Sampled Representations of Variational Autoencoders](https://mlsanity.com/paper/2109.12679/) (2021)
- [Gromov-Wasserstein Autoencoders](https://mlsanity.com/paper/2209.07007/) (2022)
- [Controlling Posterior Collapse by an Inverse Lipschitz Constraint on the Decoder Network](https://mlsanity.com/paper/2304.12770/) (2023)
- [Beyond Vanilla Variational Autoencoders: Detecting Posterior Collapse in Conditional and Hierarchical Variational Autoencoders](https://mlsanity.com/paper/2306.05023/) (2023)
- [Competency Assessment for Autonomous Agents using Deep Generative Models](https://mlsanity.com/paper/2203.12670/) (2022)
- [Matching aggregate posteriors in the variational autoencoder](https://mlsanity.com/paper/2311.07693/) (2023)
- [XPROAX-Local explanations for text classification with progressive neighborhood approximation](https://mlsanity.com/paper/2109.15004/) (2021)
- [Group Activity Recognition in Basketball Tracking Data -- Neural Embeddings in Team Sports (NETS)](https://mlsanity.com/paper/2209.00451/) (2022)
- [FONDUE: an algorithm to find the optimal dimensionality of the latent representations of variational autoencoders](https://mlsanity.com/paper/2209.12806/) (2022)
- [CHIMLE: Conditional Hierarchical IMLE for Multimodal Conditional Image Synthesis](https://mlsanity.com/paper/2211.14286/) (2022)
- [Posterior Collapse as a Phase Transition in Variational Autoencoders](https://mlsanity.com/paper/2510.01621/) (2025)
- [Covariate-informed Representation Learning to Prevent Posterior Collapse of iVAE](https://mlsanity.com/paper/2202.04206/) (2022)
- [Variational Meta Reinforcement Learning for Social Robotics](https://mlsanity.com/paper/2206.03211/) (2022)
- [Learning to Drop Out: An Adversarial Approach to Training Sequence VAEs](https://mlsanity.com/paper/2209.12590/) (2022)
- [How good are variational autoencoders at transfer learning?](https://mlsanity.com/paper/2304.10767/) (2023)
- [Bayesian Transfer Learning](https://mlsanity.com/paper/2312.13484/) (2023)
- [On the Convergence Analysis of Over-Parameterized Variational Autoencoders: A Neural Tangent Kernel Perspective](https://mlsanity.com/paper/2409.05349/) (2024)
- [Top-down inference in an early visual cortex inspired hierarchical Variational Autoencoder](https://mlsanity.com/paper/2206.00436/) (2022)

## Abstract (arXiv)

In narrow asymptotic settings Gaussian VAE models of continuous data have been shown to possess global optima aligned with ground-truth distributions. Even so, it is well known that poor solutions whereby the latent posterior collapses to an uninformative prior are sometimes obtained in practice. However, contrary to conventional wisdom that largely assigns blame for this phenomena on the undue influence of KL-divergence regularization, we will argue that posterior collapse is, at least in part, a direct consequence of bad local minima inherent to the loss surface of deep autoencoder networks. In particular, we prove that even small nonlinear perturbations of affine VAE decoder models can produce such minima, and in deeper models, analogous minima can force the VAE to behave like an aggressive truncation operator, provably discarding information along all latent dimensions in certain circumstances. Regardless, the underlying message here is not meant to undercut valuable existing explanations of posterior collapse, but rather, to refine the discussion and elucidate alternative risk factors that may have been previously underappreciated.