mlsanityData to 7 Oct 2026

Don't Blame the ELBO! A Linear VAE Perspective on Posterior Collapse

arXiv 1911.02469 ↗First posted 6 Nov 2019cs.LG stat.MLcited by 120

Not on the map. Why?

Abstract (arXiv)

Posterior collapse in Variational Autoencoders (VAEs) arises when the variational posterior distribution closely matches the prior for a subset of latent variables. This paper presents a simple and intuitive explanation for posterior collapse through the analysis of linear VAEs and their direct correspondence with Probabilistic PCA (pPCA). We explain how posterior collapse may occur in pPCA due to local maxima in the log marginal likelihood. Unexpectedly, we prove that the ELBO objective for the linear VAE does not introduce additional spurious local maxima relative to log marginal likelihood. We show further that training a linear VAE with exact variational inference recovers an identifiable global maximum corresponding to the principal component directions. Empirically, we find that our linear analysis is predictive even for high-capacity, non-linear VAEs and helps explain the relationship between the observation noise, local maxima, and posterior collapse in deep Gaussian VAEs.

Key facts. Cited by 120 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Nov 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 17; 2022: 27; 2023: 20; 2024: 15; 2025: 18; 2026: 23.

120 papers.

Sort:

By area:

  1. 2021HuMoR: 3D Human Motion Model for Robust Pose Estimationcited by 319
  2. 2023Autoregressive Diffusion Model for Graph Generationcited by 82
  3. 2023Posterior Collapse and Latent Variable Non-identifiabilitycited by 57
  4. 2022Embrace the Gap: VAEs Perform Independent Mechanism Analysiscited by 28
  5. 2023Symmetry Induces Structure and Constraint of Learningcited by 25
  6. 2025Latent Thought Models with Variational Bayes Inference-Time Computationcited by 25
  7. 2021Solving Inverse Problems by Joint Posterior Maximization with Autoencoding Priorcited by 24
  8. 2022Exact Solutions of a Deep Linear Networkcited by 24
  9. 2021Bi-level Feature Alignment for Versatile Image Translation and Manipulationcited by 20
  10. 2022Marginal Contrastive Correspondence for Guided Image Generationcited by 19
  11. 2022ContrastVAE: Contrastive Variational AutoEncoder for Sequential Recommendationcited by 19
  12. 2022What shapes the loss landscape of self-supervised learning?cited by 19
38 more
  1. 2023Complexity Matters: Rethinking the Latent Space for Generative Modelingcited by 19
  2. 2022Posterior Collapse of a Linear Latent Variable Modelcited by 18
  3. 2022Multi-Rate VAE: Train Once, Get the Full Rate-Distortion Curvecited by 18
  4. 2021Euro-PVI: Pedestrian Vehicle Interactions in Dense Urban Centerscited by 16
  5. 2021Preventing Oversmoothing in VAE via Generalized Variance Parameterizationcited by 15
  6. 2023Learning Energy-Based Prior Model with Diffusion-Amortized MCMCcited by 13
  7. 2021VAE Approximation Error: ELBO and Exponential Familiescited by 12
  8. 2023Learning Dynamics in Linear VAE: Posterior Collapse Threshold, Superfluous Latent Space Pitfalls, and Speedup with KL Annealingcited by 12
  9. 2021Variational autoencoders in the presence of low-dimensional data: landscape and implicit biascited by 11
  10. 2023Revisiting Structured Variational Autoencoderscited by 11
  11. 2024LLM Reading Tea Leaves: Automatically Evaluating Topic Models with Large Language Modelscited by 11
  12. 2022How do Variational Autoencoders Learn? Insights from Representational Similaritycited by 10
  13. 2023Event Temporal Relation Extraction with Bayesian Translational Modelcited by 10
  14. 2022Auto-regressive Image Synthesis with Integrated Quantizationcited by 9
  15. 2023Distributional Learning of Variational AutoEncoder: Application to Synthetic Data Generationcited by 9
  16. 2024Remove Symmetries to Control Model Expressivity and Improve Optimizationcited by 8
  17. 2025Augmented Deep Contexts for Spatially Embedded Video Codingcited by 8
  18. 2021Be More Active! Understanding the Differences between Mean and Sampled Representations of Variational Autoencoderscited by 7
  19. 2023Controlling Posterior Collapse by an Inverse Lipschitz Constraint on the Decoder Networkcited by 7
  20. 2022Competency Assessment for Autonomous Agents using Deep Generative Modelscited by 6
  21. 2022Variational Sparse Coding with Learned Thresholdingcited by 6
  22. 2022PVGRU: Generating Diverse and Relevant Dialogue Responses via Pseudo-Variational Mechanismcited by 6
  23. 2023Beyond Vanilla Variational Autoencoders: Detecting Posterior Collapse in Conditional and Hierarchical Variational Autoencoderscited by 6
  24. 2023InVAErt networks: a data-driven framework for model synthesis and identifiability analysiscited by 6
  25. 2023Matching aggregate posteriors in the variational autoencodercited by 6
  26. 2025ARD-VAE: A Statistical Formulation to Find the Relevant Latent Dimensions of Variational Autoencoderscited by 6
  27. 2021Demystifying Inductive Biases for β-VAE Based Architecturescited by 5
  28. 2022Exact Phase Transitions in Deep Learningcited by 5
  29. 2022FONDUE: an algorithm to find the optimal dimensionality of the latent representations of variational autoencoderscited by 5
  30. 2023On Masked Pre-training and the Marginal Likelihoodcited by 5
  31. 2023High-dimensional Asymptotics of VAEs: Threshold of Posterior Collapse and Dataset-Size Dependence of Rate-Distortion Curvecited by 5
  32. 2024Functional Imaging Constrained Diffusion for Brain PET Synthesis from Structural MRIcited by 5
  33. 2026Swap-guided Preference Learning for Personalized Reinforcement Learning from Human Feedbackcited by 5
  34. 2021EXoN: EXplainable encoder Networkcited by 4
  35. 2021Discriminative Multimodal Learning via Conditional Priors in Generative Modelscited by 4
  36. 2022Split Hierarchical Variational Compressioncited by 4
  37. 2022CHIMLE: Conditional Hierarchical IMLE for Multimodal Conditional Image Synthesiscited by 4
  38. 2025Unsupervised learning for anticipating critical transitionscited by 4

50 most cited of 120 All 120 (JSON)