mlsanityData to 7 Oct 2026

Preventing Oversmoothing in VAE via Generalized Variance Parameterization

arXiv 2102.08663 ↗First posted 17 Feb 2021cs.LG cs.CVcited by 15

Not on the map. Why?

Abstract (arXiv)

Variational autoencoders (VAEs) often suffer from posterior collapse, which is a phenomenon in which the learned latent space becomes uninformative. This is often related to the hyperparameter resembling the data variance. It can be shown that an inappropriate choice of this hyperparameter causes the oversmoothness in the linearly approximated case and can be empirically verified for the general cases. Moreover, determining such appropriate choice becomes infeasible if the data variance is non-uniform or conditional. Therefore, we propose VAE extensions with generalized parameterizations of the data variance and incorporate maximum likelihood estimation into the objective function to adaptively regularize the decoder smoothness. The images generated from proposed VAE extensions show improved Fr'echet inception distance (FID) on MNIST and CelebA datasets.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Builds on 30 of its 39 references that are in the map. First posted on arXiv 17 Feb 2021.

Builds on

30 of 39 references.

References · 30

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  3. 2017Neural Discrete Representation Learningcited by 4,650
  4. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  5. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  6. 2017Improved Training of Wasserstein GANscited by 2,223
  7. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  8. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  9. 2016InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Netscited by 858
  10. 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 636
  11. 2020NVAE: A Deep Hierarchical Variational Autoencodercited by 635
  12. 2015Generating Sentences from a Continuous Spacecited by 607
18 more
  1. 2015Importance Weighted Autoencoderscited by 454
  2. 2017PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modificationscited by 415
  3. 2017Wasserstein Auto-Encoderscited by 371
  4. 2015Adversarial Autoencoderscited by 359
  5. 2018How Does Batch Normalization Help Optimization?cited by 355
  6. 2016Ladder Variational Autoencoderscited by 315
  7. 2020Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Imagescited by 261
  8. 2019Diagnosing and Enhancing VAE Modelscited by 192
  9. 2019Cyclical Annealing Schedule: A Simple Approach to Mitigating KL Vanishingcited by 191
  10. 2019From Variational to Deterministic Autoencoderscited by 131
  11. 2019Lagging Inference Networks and Posterior Collapse in Variational Autoencoderscited by 124
  12. 2019Don't Blame the ELBO! A Linear VAE Perspective on Posterior Collapsecited by 120
  13. 2019Preventing Posterior Collapse with delta-VAEscited by 86
  14. 2018Spherical Latent Spaces for Stable Variational Autoencoderscited by 66
  15. 2019The Usual Suspects? Reassessing Blame for VAE Posterior Collapsecited by 52
  16. 2019MAE: Mutual Posterior-Divergence Regularization for Variational AutoEncoderscited by 15
  17. 2020Variational Autoencoders with Normalizing Flow Decoderscited by 7
  18. 2020ControlVAE: Controllable Variational Autoencodercited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 3; 2024: 3; 2025: 4; 2026: 4.

15 papers.

Sort:

By area:

  1. 2023HQ-VAE: Hierarchical Discrete Representation Learning with Variational Bayescited by 27
  2. 2024PID: Physics-Informed Diffusion Model for Infrared Image Generationcited by 17
  3. 2022Amortized Variational Inference: A Systematic Reviewcited by 16
  4. 2025VCT: Training Consistency Models with Variational Noise Couplingcited by 15
  5. 2025Flow Matching Meets Biology and Life Science: A Surveycited by 14
  6. 2024Learning Physics for Unveiling Hidden Earthquake Ground Motions via Conditional Generative Modelingcited by 7
  7. 2023How to train your VAEcited by 3
  8. 2025Denoising Multi-Beta VAE: Representation Learning for Disentanglement and Generationcited by 3
  9. 2023Causal Dynamic Variational Autoencoder for Counterfactual Regression in Longitudinal Datacited by 1
  10. 2026Reasoning-guided Collaborative Filtering with Language Models for Explainable Recommendationcited by 1
  11. 2024Teacher-free Latent Self-distillation and Class-separable Representations for Lightweight IoT Attack Detectioncited by 0
  12. 2025DiffKnock: Diffusion-based Knockoff Statistics for Neural Networks Inferencecited by 0
3 more
  1. 2026Mutual Information Collapse Explains Disentanglement Failure in β-VAEscited by 0
  2. 2026No Free Lunch for Synthetic Images under Data Scarcity Conditionscited by 0
  3. 2026K-IPO: Kendall-constrained Importance Preserving Oversampling for Imbalanced Tabular Datacited by 0