mlsanityData to 7 Oct 2026

Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Images

arXiv 2011.10650 ↗First posted 20 Nov 2020cs.LG cs.CVcited by 261

Not on the map. Why?

Cited by 65 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We present a hierarchical VAE that, for the first time, generates samples quickly while outperforming the PixelCNN in log-likelihood on all natural image benchmarks. We begin by observing that, in theory, VAEs can actually represent autoregressive models, as well as faster, better models if they exist, when made sufficiently deep. Despite this, autoregressive models have historically outperformed VAEs in log-likelihood. We test if insufficient depth explains why by scaling a VAE to greater stochastic depth than previously explored and evaluating it CIFAR-10, ImageNet, and FFHQ. In comparison to the PixelCNN, these very deep VAEs achieve higher likelihoods, use fewer parameters, generate samples thousands of times faster, and are more easily applied to high-resolution images. Qualitative studies suggest this is because the VAE learns efficient hierarchical visual representations. We release our source code and models at https://github.com/openai/vdvae.

Key facts. Cited by 261 papers in the map (data through 7 Oct 2026). First posted on arXiv 20 Nov 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 49; 2022: 59; 2023: 64; 2024: 42; 2025: 33; 2026: 14.

261 papers.

Sort:

By area:

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  2. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  3. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  4. 2022Flow Matching for Generative Modelingcited by 5,426
  5. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  6. 2023Mastering Diverse Domains through World Modelscited by 1,193
  7. 2021Variational Diffusion Modelscited by 1,145
  8. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,107
  9. 2022Autoregressive Image Generation using Residual Quantizationcited by 706
  10. 2021Maximum Likelihood Training of Score-Based Diffusion Modelscited by 670
  11. 2021Vector-quantized Image Modeling with Improved VQGANcited by 656
  12. 2021VideoGPT: Video Generation using VQ-VAE and Transformerscited by 651
38 more
  1. 2021Score-based Generative Modeling in Latent Spacecited by 634
  2. 2021Label-Efficient Semantic Segmentation with Diffusion Modelscited by 495
  3. 2021Knowledge Distillation in Iterative Generative Models for Improved Sampling Speedcited by 344
  4. 2022Perception Prioritized Training of Diffusion Modelscited by 301
  5. 2022The Vendi Score: A Diversity Evaluation Metric for Machine Learningcited by 299
  6. 2022Flexible Diffusion Modeling of Long Videoscited by 294
  7. 2021Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Modelscited by 251
  8. 2022Long Video Generation with Time-Agnostic VQGAN and Time-Sensitive Transformercited by 244
  9. 2021Improving Robustness using Generated Datacited by 233
  10. 2021Likelihood Training of Schr"odinger Bridge using Forward-Backward SDEs Theorycited by 216
  11. 2021Autoregressive Diffusion Modelscited by 208
  12. 2021Fixing Data Augmentation to Improve Adversarial Robustnesscited by 201
  13. 2023Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion modelscited by 199
  14. 2022Generative Modelling With Inverse Heat Dissipationcited by 164
  15. 2023Reconstructing the Mind's Eye: fMRI-to-Image with Contrastive Learning and Diffusion Priorscited by 156
  16. 2021Learning to Efficiently Sample from Diffusion Probabilistic Modelscited by 129
  17. 2021ImageBART: Bidirectional Context with Multinomial Diffusion for Autoregressive Image Synthesiscited by 122
  18. 2022Generating Long Videos of Dynamic Scenescited by 113
  19. 2022DiffuseVAE: Efficient, Controllable and High-Fidelity Generation from Low-Dimensional Latentscited by 103
  20. 2022GFlowNets and variational inferencecited by 103
  21. 2023Natural scene reconstruction from fMRI signals using generative latent diffusioncited by 97
  22. 2021D2C: Diffusion-Denoising Models for Few-shot Conditional Generationcited by 96
  23. 2024Normalizing Flows are Capable Generative Modelscited by 89
  24. 2023Minimizing Trajectory Curvature of ODE-based Generative Modelscited by 87
  25. 2021Multimodal Conditional Image Synthesis with Product-of-Experts GANscited by 75
  26. 2021Predictive Coding: a Theoretical and Experimental Reviewcited by 74
  27. 2021Unleashing Transformers: Parallel Token Prediction with Discrete Absorbing Diffusion for Fast High-Resolution Image Generation from Vector-Quantized Codescited by 71
  28. 2023Bayesian Flow Networkscited by 62
  29. 2023Towards Accurate Image Coding: Improved Autoregressive Image Generation with Dynamic Vector Quantizationcited by 59
  30. 2021Hierarchical VAEs Know What They Don't Knowcited by 58
  31. 2023High Fidelity Image Counterfactuals with Probabilistic Causal Modelscited by 53
  32. 2023Aligning Optimization Trajectories with Diffusion Models for Constrained Design Generationcited by 51
  33. 2024Deep Generative Models through the Lens of the Manifold Hypothesis: A Survey and New Connectionscited by 47
  34. 2023DREAM: Visual Decoding from Reversing Human Visual Systemcited by 44
  35. 2025DC-AE 1.5: Accelerating Diffusion Model Convergence with Structured Latent Spacecited by 44
  36. 2022Long Document Summarization with Top-down and Bottom-up Inferencecited by 42
  37. 2022Maximum Likelihood Training of Implicit Nonlinear Diffusion Modelscited by 42
  38. 2021Clockwork Variational Autoencoderscited by 40

50 most cited of 261 All 261 (JSON)