mlsanityData to 7 Oct 2026

Preventing Posterior Collapse with delta-VAEs

arXiv 1901.03416 ↗First posted 10 Jan 2019cs.LG stat.MLcited by 86

Not on the map. Why?

Abstract (arXiv)

Due to the phenomenon of "posterior collapse," current latent variable generative models pose a challenging design choice that either weakens the capacity of the decoder or requires augmenting the objective so it does not only maximize the likelihood of the data. In this paper, we propose an alternative that utilizes the most powerful generative models as decoders, whilst optimising the variational lower bound all while ensuring that the latent variables preserve and encode useful information. Our proposed δ-VAEs achieve this by constraining the variational family for the posterior to have a minimum distance to the prior. For sequential latent variable models, our approach resembles the classic representation learning approach of slow feature analysis. We demonstrate the efficacy of our approach at modeling text on LM1B and modeling images: learning representations, improving sample quality, and achieving state of the art log-likelihood on CIFAR-10 and ImageNet 32× 32.

Key facts. Cited by 86 papers in the map (data through 7 Oct 2026). First posted on arXiv 10 Jan 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 25; 2022: 24; 2023: 15; 2024: 6; 2025: 8; 2026: 8.

86 papers.

Sort:

By area:

  1. 2021Maximum Likelihood Training of Score-Based Diffusion Modelscited by 670
  2. 2021Score-based Generative Modeling in Latent Spacecited by 634
  3. 2021Step-unrolled Denoising Autoencoders for Text Generationcited by 102
  4. 2023Posterior Collapse and Latent Variable Non-identifiabilitycited by 57
  5. 2022FLAG: Flow-based 3D Avatar Generation from Sparse Observationscited by 54
  6. 2021Densely connected normalizing flowscited by 49
  7. 2022Maximum Likelihood Training of Implicit Nonlinear Diffusion Modelscited by 42
  8. 2022Dataset Condensation with Latent Space Knowledge Factorization and Sharingcited by 34
  9. 2021A Critical Look at the Consistency of Causal Estimation With Deep Latent Variable Modelscited by 29
  10. 2021Data-Driven Offline Optimization For Architecting Hardware Acceleratorscited by 29
  11. 2022Efficient-VDVAE: Less is morecited by 26
  12. 2023AI-Generated Content (AIGC) for Various Data Modalities: A Surveycited by 25
38 more
  1. 2021Variable-rate discrete representation learningcited by 24
  2. 2022The Free Energy Principle for Perception and Action: A Deep Learning Perspectivecited by 24
  3. 2021Sawtooth Factorial Topic Embeddings Guided Gamma Belief Networkcited by 23
  4. 2021A survey on Variational Autoencoders from a GreenAI perspectivecited by 22
  5. 2022Fuse It More Deeply! A Variational Transformer with Layer-Wise Latent Variable Inference for Text Generationcited by 20
  6. 2022Missing Data Imputation and Acquisition with Deep Hierarchical Models and Hamiltonian Monte Carlocited by 19
  7. 2023Normalizing Flow-based Neural Process for Few-Shot Knowledge Graph Completioncited by 19
  8. 2023Complexity Matters: Rethinking the Latent Space for Generative Modelingcited by 19
  9. 2026Unified Latents (UL): How to train your latentscited by 19
  10. 2022Posterior Collapse of a Linear Latent Variable Modelcited by 18
  11. 2022Multi-Rate VAE: Train Once, Get the Full Rate-Distortion Curvecited by 18
  12. 2022Amortized Variational Inference: A Systematic Reviewcited by 16
  13. 2021Preventing Oversmoothing in VAE via Generalized Variance Parameterizationcited by 15
  14. 2022Hierarchical Latent Structure for Multi-Modal Vehicle Trajectory Forecastingcited by 14
  15. 2022Modeling Multimodal Aleatoric Uncertainty in Segmentation with Mixture of Stochastic Expertscited by 13
  16. 2023Learning Energy-Based Prior Model with Diffusion-Amortized MCMCcited by 13
  17. 2024A Gray-box Attack against Latent Diffusion Model-based Image Editing by Posterior Collapsecited by 13
  18. 2021Variational autoencoders in the presence of low-dimensional data: landscape and implicit biascited by 11
  19. 2022Comparing the latent space of generative modelscited by 11
  20. 2022Improving Small Molecule Generation using Mutual Information Machinecited by 10
  21. 2021Conditional Temporal Variational AutoEncoder for Action Video Predictioncited by 9
  22. 2025FerretNet: Efficient Synthetic Image Detection via Local Pixel Dependenciescited by 9
  23. 2023Controlling Posterior Collapse by an Inverse Lipschitz Constraint on the Decoder Networkcited by 7
  24. 2021Regularizing Variational Autoencoder with Diversity and Uncertainty Awarenesscited by 6
  25. 2022Forget-me-not! Contrastive Critics for Mitigating Posterior Collapsecited by 6
  26. 2023Beyond Vanilla Variational Autoencoders: Detecting Posterior Collapse in Conditional and Hierarchical Variational Autoencoderscited by 6
  27. 2023InVAErt networks: a data-driven framework for model synthesis and identifiability analysiscited by 6
  28. 2024Entropy-Informed Weighting Channel Normalizing Flow for Deep Generative Modelscited by 5
  29. 2025Distribution Matching Variational AutoEncodercited by 5
  30. 2021EXoN: EXplainable encoder Networkcited by 4
  31. 2021On the Generative Utility of Cyclic Conditionalscited by 4
  32. 2022Split Hierarchical Variational Compressioncited by 4
  33. 2022AS-IntroVAE: Adversarial Similarity Distance Makes Robust IntroVAEcited by 4
  34. 2023SC-VAE: Sparse Coding-based Variational Autoencoder with Learned ISTAcited by 4
  35. 2024A Non-negative VAE:the Generalized Gamma Belief Networkcited by 4
  36. 2021Variational Nested Dropoutcited by 3
  37. 2022Covariate-informed Representation Learning to Prevent Posterior Collapse of iVAEcited by 3
  38. 2022Learning to Drop Out: An Adversarial Approach to Training Sequence VAEscited by 3

50 most cited of 86 All 86 (JSON)