mlsanityData to 7 Oct 2026

Discouraging posterior collapse in hierarchical Variational Autoencoders using context

arXiv 2302.09976 ↗First posted 20 Feb 2023cs.LG cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

Hierarchical Variational Autoencoders (VAEs) are among the most popular likelihood-based generative models. There is a consensus that the top-down hierarchical VAEs allow effective learning of deep latent structures and avoid problems like posterior collapse. Here, we show that this is not necessarily the case, and the problem of collapsing posteriors remains. To discourage this issue, we propose a deep hierarchical VAE with a context on top. Specifically, we use a Discrete Cosine Transform to obtain the last latent variable. In a series of experiments, we observe that the proposed modification allows us to achieve better utilization of the latent space and does not harm the model's generative abilities.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 29 of its 45 references that are in the map. First posted on arXiv 20 Feb 2023.

Builds on

29 of 45 references.

References · 29

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  3. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  4. 2017Neural Discrete Representation Learningcited by 4,650
  5. 2021Variational Diffusion Modelscited by 1,145
  6. 2016Automatic chemical design using a data-driven continuous representation of moleculescited by 762
  7. 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 636
  8. 2020NVAE: A Deep Hierarchical Variational Autoencodercited by 635
  9. 2021Score-based Generative Modeling in Latent Spacecited by 634
  10. 2015Generating Sentences from a Continuous Spacecited by 607
  11. 2015Importance Weighted Autoencoderscited by 454
  12. 2018Do Deep Generative Models Know What They Don't Know?cited by 443
17 more
  1. 2016Ladder Variational Autoencoderscited by 315
  2. 2020Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Imagescited by 261
  3. 2017VAE with a VampPriorcited by 247
  4. 2019Cyclical Annealing Schedule: A Simple Approach to Mitigating KL Vanishingcited by 191
  5. 2021A Variational Perspective on Diffusion-Based Generative Models and Score Matchingcited by 182
  6. 2019Neural Stochastic Differential Equations: Deep Latent Gaussian Models in the Diffusion Limitcited by 168
  7. 2019Lagging Inference Networks and Posterior Collapse in Variational Autoencoderscited by 124
  8. 2019BIVA: A Very Deep Hierarchy of Latent Variables for Generative Modelingcited by 109
  9. 2019Preventing Posterior Collapse with delta-VAEscited by 86
  10. 2023Posterior Collapse and Latent Variable Non-identifiabilitycited by 57
  11. 2021Consistency Regularization for Variational Auto-Encoderscited by 50
  12. 2018Avoiding Latent Variable Collapse With Generative Skip Modelscited by 47
  13. 2021Diffusion Priors In Variational Autoencoderscited by 25
  14. 2021Invertible DenseNets with Concatenated LipSwishcited by 17
  15. 2020Preventing Posterior Collapse with Levenshtein Variational Autoencodercited by 12
  16. 2022Alleviating Adversarial Attacks on Variational Autoencoders with MCMCcited by 10
  17. 2017Semi-Supervised Generation with Cluster-aware Generative Modelscited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Synthetic Flight Data Generation Using Generative Modelscited by 0