mlsanityData to 7 Oct 2026

Posterior Collapse of a Linear Latent Variable Model

arXiv 2205.04009 ↗First posted 9 May 2022cs.LG stat.MLcited by 18

Not on the map. Why?

Abstract (arXiv)

This work identifies the existence and cause of a type of posterior collapse that frequently occurs in the Bayesian deep learning practice. For a general linear latent variable model that includes linear variational autoencoders as a special case, we precisely identify the nature of posterior collapse to be the competition between the likelihood and the regularization of the mean due to the prior. Our result suggests that posterior collapse may be related to neural collapse and dimensional collapse and could be a subclass of a general problem of learning for deeper architectures.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 21 of its 50 references that are in the map. First posted on arXiv 9 May 2022.

Builds on

21 of 50 references.

References · 21

  1. 2013Exact solutions to the nonlinear dynamics of learning in deep linear neural networkscited by 748
  2. 2020Prevalence of Neural Collapse during the terminal phase of deep learning trainingcited by 713
  3. 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 636
  4. 2015Generating Sentences from a Continuous Spacecited by 607
  5. 2019Surprises in High-Dimensional Ridgeless Least Squares Interpolationcited by 508
  6. 2019Implicit Regularization in Deep Matrix Factorizationcited by 402
  7. 2021Understanding Dimensional Collapse in Contrastive Self-supervised Learningcited by 374
  8. 2016Deep Learning without Poor Local Minimacited by 233
  9. 2019Don't Blame the ELBO! A Linear VAE Perspective on Posterior Collapsecited by 120
  10. 2016Identity Matters in Deep Learningcited by 107
  11. 2019Preventing Posterior Collapse with delta-VAEscited by 86
  12. 2017Deep linear neural networks with arbitrary loss: All local minima are globalcited by 64
9 more
  1. 2020Generative Neurosymbolic Machinescited by 57
  2. 2019The Usual Suspects? Reassessing Blame for VAE Posterior Collapsecited by 52
  3. 2017Depth Creates No Bad Local Minimacited by 29
  4. 2018Improving Explorability in Variational Inference with Annealed Variational Objectivescited by 24
  5. 2022Exact Solutions of a Deep Linear Networkcited by 24
  6. 2018The loss surface of deep linear networks viewed through the algebraic geometry lenscited by 20
  7. 2021VAE Approximation Error: ELBO and Exponential Familiescited by 12
  8. 2018Regularized Singular Value Decomposition and Application to Recommender Systemcited by 6
  9. 2022Stochastic Neural Networks with Infinite Width are Deterministiccited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 7; 2024: 3; 2025: 1; 2026: 5.

18 papers.

Sort:

By area:

  1. 2023Symmetry Induces Structure and Constraint of Learningcited by 25
  2. 2022What shapes the loss landscape of self-supervised learning?cited by 19
  3. 2024Parameter Symmetry and Noise Equilibrium of Stochastic Gradient Descentcited by 19
  4. 2022Multi-Rate VAE: Train Once, Get the Full Rate-Distortion Curvecited by 18
  5. 2023Learning Dynamics in Linear VAE: Posterior Collapse Threshold, Superfluous Latent Space Pitfalls, and Speedup with KL Annealingcited by 12
  6. 2023Type-II Saddles and Probabilistic Stability of Stochastic Gradient Descentcited by 8
  7. 2024Remove Symmetries to Control Model Expressivity and Improve Optimizationcited by 8
  8. 2023Controlling Posterior Collapse by an Inverse Lipschitz Constraint on the Decoder Networkcited by 7
  9. 2023Beyond Vanilla Variational Autoencoders: Detecting Posterior Collapse in Conditional and Hierarchical Variational Autoencoderscited by 6
  10. 2023High-dimensional Asymptotics of VAEs: Threshold of Posterior Collapse and Dataset-Size Dependence of Rate-Distortion Curvecited by 5
  11. 2025Posterior Collapse as a Phase Transition in Variational Autoencoderscited by 4
  12. 2023Bayesian Transfer Learningcited by 3
6 more
  1. 2024Preventing Model Collapse in Gaussian Process Latent Variable Modelscited by 3
  2. 2026Iterative Amortized Hierarchical VAEcited by 0
  3. 2026Mutual Information Collapse Explains Disentanglement Failure in β-VAEscited by 0
  4. 2026Entropy-Based Characterisation of the Polarised Regime in Latent Variable Modelscited by 0
  5. 2026Feature Lottery? A Bifurcation Theory of Concept Emergencecited by 0
  6. 2026Latent evolving World Action Modelcited by 0