mlsanityData to 7 Oct 2026

Optimizing Hierarchical Image VAEs for Sample Quality

arXiv 2210.10205 ↗First posted 18 Oct 2022cs.LG cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

While hierarchical variational autoencoders (VAEs) have achieved great density estimation on image modeling tasks, samples from their prior tend to look less convincing than models with similar log-likelihood. We attribute this to learned representations that over-emphasize compressing imperceptible details of the image. To address this, we introduce a KL-reweighting strategy to control the amount of infor mation in each latent group, and employ a Gaussian output layer to reduce sharpness in the learning objective. To trade off image diversity for fidelity, we additionally introduce a classifier-free guidance strategy for hierarchical VAEs. We demonstrate the effectiveness of these techniques in our experiments. Code is available at https://github.com/tcl9876/visual-vae.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 30 of its 35 references that are in the map. First posted on arXiv 18 Oct 2022.

Builds on

30 of 35 references.

References · 30

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  6. 2022Classifier-Free Diffusion Guidancecited by 5,717
  7. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  8. 2013Auto-Encoding Variational Bayescited by 3,980
  9. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  10. 2015Variational Inference with Normalizing Flowscited by 2,161
  11. 2019Generating Long Sequences with Sparse Transformerscited by 1,523
  12. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
18 more
  1. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,287
  2. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155
  3. 2021Variational Diffusion Modelscited by 1,145
  4. 2019Improved Precision and Recall Metric for Assessing Generative Modelscited by 925
  5. 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 636
  6. 2020NVAE: A Deep Hierarchical Variational Autoencodercited by 635
  7. 2017PixelCNN++: Improving the PixelCNN with Discretized Logistic Mixture Likelihood and Other Modificationscited by 415
  8. 2016Ladder Variational Autoencoderscited by 315
  9. 2020Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Imagescited by 261
  10. 2018A Note on the Inception Scorecited by 225
  11. 2019Diagnosing and Enhancing VAE Modelscited by 192
  12. 2017Spectral Norm Regularization for Improving the Generalizability of Deep Learningcited by 178
  13. 2016Variational Lossy Autoencodercited by 175
  14. 2019BIVA: A Very Deep Hierarchy of Latent Variables for Generative Modelingcited by 109
  15. 2021D2C: Diffusion-Denoising Models for Few-shot Conditional Generationcited by 96
  16. 2018DVAE++: Discrete Variational Autoencoders with Overlapping Transformationscited by 31
  17. 2022Efficient-VDVAE: Less is morecited by 26
  18. 2021Improved Autoregressive Modeling with Distribution Smoothingcited by 18

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 2; 2025: 1.

5 papers.

  1. 2025Scale-wise Distillation of Diffusion Modelscited by 11
  2. 2023Inverse problem regularization with hierarchical variational autoencoderscited by 6
  3. 2023High Fidelity Image Synthesis With Deep VAEs In Latent Spacecited by 4
  4. 2024Scaling Image Tokenizers with Grouped Spherical Quantizationcited by 4
  5. 2024Nested Diffusion Models Using Hierarchical Latent Priorscited by 2