mlsanityData to 14 Oct 2026

High Fidelity Image Synthesis With Deep VAEs In Latent Space

arXiv 2303.13714 ↗First posted 23 Mar 2023cs.CV cs.LG eess.IVcited by 4

Not on the map. Why?

Abstract (arXiv)

We present fast, realistic image generation on high-resolution, multimodal datasets using hierarchical variational autoencoders (VAEs) trained on a deterministic autoencoder's latent space. In this two-stage setup, the autoencoder compresses the image into its semantic features, which are then modeled with a deep VAE. With this method, the VAE avoids modeling the fine-grained details that constitute the majority of the image's code length, allowing it to focus on learning its structural components. We demonstrate the effectiveness of our two-stage approach, achieving a FID of 9.34 on the ImageNet-256 dataset which is comparable to BigGAN. We make our implementation available online.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 37 of its 44 references that are in the map. First posted on arXiv 23 Mar 2023.

Builds on

37 of 44 references.

References · 37

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  7. 2022Classifier-Free Diffusion Guidancecited by 5,847
  8. 2013Auto-Encoding Variational Bayescited by 4,053
  9. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  10. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  11. 2016Improved Techniques for Training GANscited by 2,970
  12. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
25 more
  1. 2015Variational Inference with Normalizing Flowscited by 2,170
  2. 2016Density estimation using Real NVPcited by 2,042
  3. 2021Image Super-Resolution via Iterative Refinementcited by 1,511
  4. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,387
  5. 2021Palette: Image-to-Image Diffusion Modelscited by 1,300
  6. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,295
  7. 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,163
  8. 2021Vector-quantized Image Modeling with Improved VQGANcited by 658
  9. 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 640
  10. 2020NVAE: A Deep Hierarchical Variational Autoencodercited by 637
  11. 2021Score-based Generative Modeling in Latent Spacecited by 636
  12. 2022StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasetscited by 492
  13. 2022Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priorscited by 487
  14. 2021SRDiff: Single Image Super-Resolution with Diffusion Probabilistic Modelscited by 415
  15. 2015Autoencoding beyond pixels using a learned similarity metriccited by 407
  16. 2016Ladder Variational Autoencoderscited by 318
  17. 2021The Intrinsic Dimension of Images and Its Impact on Learningcited by 315
  18. 2020Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Imagescited by 264
  19. 2019Diagnosing and Enhancing VAE Modelscited by 193
  20. 2017Spectral Norm Regularization for Improving the Generalizability of Deep Learningcited by 182
  21. 2016Variational Lossy Autoencodercited by 176
  22. 2019BIVA: A Very Deep Hierarchy of Latent Variables for Generative Modelingcited by 109
  23. 2022Efficient-VDVAE: Less is morecited by 26
  24. 2021Conditional Image Generation by Conditioning Variational Auto-Encoderscited by 25
  25. 2022Optimizing Hierarchical Image VAEs for Sample Qualitycited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 1; 2026: 1.

4 papers.

  1. 2024Assessing Image Quality Using a Simple Generative Representationcited by 1
  2. 2025ADMN: A Layer-Wise Adaptive Multimodal Network for Dynamic Input Noise and Compute Resourcescited by 1
  3. 2024StableSemantics: A Synthetic Language-Vision Dataset of Semantic Representations in Naturalistic Imagescited by 0
  4. 2026Sphere Encoder 2cited by 0