mlsanityData to 7 Oct 2026

Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models

arXiv 2103.04922 ↗First posted 8 Mar 2021cs.LG cs.CV stat.MLcited by 251

Not on the map. Why?

Cited by 63 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Deep generative models are a class of techniques that train deep neural networks to model the distribution of training samples. Research has fragmented into various interconnected approaches, each of which make trade-offs including run-time, diversity, and architectural restrictions. In particular, this compendium covers energy-based models, variational autoencoders, generative adversarial networks, autoregressive models, normalizing flows, in addition to numerous hybrid approaches. These techniques are compared and contrasted, explaining the premises behind each and how they are interrelated, while reviewing current state-of-the-art advances and implementations.

Key facts. Cited by 251 papers in the map (data through 7 Oct 2026). Builds on 202 of its 276 references that are in the map. First posted on arXiv 8 Mar 2021.

Builds on

202 of 276 references.

References · 202

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  6. 2020Denoising Diffusion Implicit Modelscited by 9,655
  7. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  8. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  9. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  10. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  11. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  12. 2017Neural Discrete Representation Learningcited by 4,650
38 more
  1. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  2. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  3. 2013Auto-Encoding Variational Bayescited by 3,980
  4. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  5. 2018Neural Ordinary Differential Equationscited by 3,733
  6. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  7. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
  8. 2016Wide Residual Networkscited by 3,057
  9. 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
  10. 2016Improved Techniques for Training GANscited by 2,938
  11. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,708
  12. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,609
  13. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  14. 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,320
  15. 2017Improved Training of Wasserstein GANscited by 2,223
  16. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
  17. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  18. 2015Variational Inference with Normalizing Flowscited by 2,161
  19. 2014Conditional Generative Adversarial Netscited by 2,152
  20. 2015Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networkscited by 2,048
  21. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,031
  22. 2016Density estimation using Real NVPcited by 2,029
  23. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 1,979
  24. 2016WaveNet: A Generative Model for Raw Audiocited by 1,710
  25. 2018Spectral Normalization for Generative Adversarial Networkscited by 1,618
  26. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,598
  27. 2020Rethinking Attention with Performerscited by 1,584
  28. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
  29. 2019Generating Long Sequences with Sparse Transformerscited by 1,523
  30. 2020Linformer: Self-Attention with Linear Complexitycited by 1,462
  31. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  32. 2019Normalizing Flows for Probabilistic Modeling and Inferencecited by 1,315
  33. 2018Demystifying MMD GANscited by 1,268
  34. 2020Training Generative Adversarial Networks with Limited Datacited by 1,184
  35. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,135
  36. 2014NICE: Non-linear Independent Components Estimationcited by 1,052
  37. 2016Least Squares Generative Adversarial Networkscited by 1,000
  38. 2016Conditional Image Generation with PixelCNN Decoderscited by 859

50 most cited of 202 All 202 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 16; 2022: 33; 2023: 62; 2024: 58; 2025: 62; 2026: 20.

251 papers.

Sort:

By area:

  1. 2022Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusioncited by 309
  2. 2022Diffusion Models for Medical Image Analysis: A Comprehensive Surveycited by 269
  3. 2022A Survey on Generative Diffusion Modelcited by 238
  4. 2023State of the Art on Diffusion Models for Visual Computingcited by 126
  5. 2021Pros and Cons of GAN Evaluation Measures: New Developmentscited by 121
  6. 2023Human Motion Generation: A Surveycited by 95
  7. 2023Generative Diffusion Models on Graphs: Methods and Applicationscited by 79
  8. 2023Synthcity: facilitating innovative use cases of synthetic data in different data modalitiescited by 76
  9. 2021Unleashing Transformers: Parallel Token Prediction with Discrete Absorbing Diffusion for Fast High-Resolution Image Generation from Vector-Quantized Codescited by 71
  10. 2022BDDM: Bilateral Denoising Diffusion Models for Fast and High-Quality Speech Synthesiscited by 64
  11. 2023LiDAR Data Synthesis with Denoising Diffusion Probabilistic Modelscited by 53
  12. 2022Diffusion-based Generative Speech Source Separationcited by 41
38 more
  1. 2023Synthetic Data for Face Recognition: Current State and Future Prospectscited by 38
  2. 2024A Comprehensive Survey on Data Augmentationcited by 38
  3. 2023Diffusion Model-Augmented Behavioral Cloningcited by 37
  4. 2023SoK: Privacy-Preserving Data Synthesiscited by 37
  5. 2023The statistical thermodynamics of generative diffusion models: Phase transitions, symmetry breaking and critical instabilitycited by 37
  6. 2021Bilateral Denoising Diffusion Modelscited by 36
  7. 2022A Study on the Evaluation of Generative Modelscited by 35
  8. 2023Beyond Privacy: Navigating the Opportunities and Challenges of Synthetic Datacited by 29
  9. 2022Diagnosing and Fixing Manifold Overfitting in Deep Generative Modelscited by 28
  10. 2022Modeling sRGB Camera Noise with Normalizing Flowscited by 28
  11. 2023Parallel Vertex Diffusion for Unified Visual Groundingcited by 28
  12. 2022Efficient-VDVAE: Less is morecited by 26
  13. 2024The Rise of Diffusion Models in Time-Series Forecastingcited by 25
  14. 2024Do Generated Data Always Help Contrastive Learning?cited by 25
  15. 2023Reimagining Synthetic Tabular Data Generation through Data-Centric AI: A Comprehensive Benchmarkcited by 23
  16. 2024scDiffusion: conditional generation of high-quality single-cell data using diffusion modelcited by 23
  17. 2021Transflower: probabilistic autoregressive dance generation with multimodal attentioncited by 22
  18. 2021Creativity and Machine Learning: A Surveycited by 21
  19. 2021Efficient Gradient Flows in Sliced-Wasserstein Spacecited by 21
  20. 2023Deep Generative Model and Its Applications in Efficient Wireless Network Management: A Tutorial and Case Studycited by 21
  21. 2023Phoenix: A Federated Generative Diffusion Modelcited by 21
  22. 2022Controllable Data Generation by Deep Learning: A Reviewcited by 20
  23. 2021A deep generative model for probabilistic energy forecasting in power systems: normalizing flowscited by 18
  24. 2022Generative Range Imaging for Learning Scene Priors of 3D LiDAR Datacited by 18
  25. 2022Multitask Brain Tumor Inpainting with Diffusion Models: A Methodological Reportcited by 18
  26. 2023The Ethical Implications of Generative Audio Models: A Systematic Literature Reviewcited by 18
  27. 2021Deep Generative Modeling for Protein Designcited by 17
  28. 2022Protein Language Models and Structure Prediction: Connection and Progressioncited by 17
  29. 2023Generative AI for Physical Layer Communications: A Surveycited by 17
  30. 2022Boomerang: Local sampling on image manifolds using diffusion modelscited by 16
  31. 2024FiTv2: Scalable and Improved Flexible Vision Transformer for Diffusion Modelcited by 15
  32. 2025Deep Learning Advancements in Anomaly Detection: A Comprehensive Surveycited by 15
  33. 2024A Practical Guide to Sample-based Statistical Distances for Evaluating Generative Models in Sciencecited by 14
  34. 2025Generative Models in Decision Making: A Surveycited by 14
  35. 2021A review and experimental evaluation of deep learning methods for MRI reconstructioncited by 13
  36. 2024Fast LiDAR Data Generation with Rectified Flowscited by 13
  37. 2021The Devil is in the GAN: Backdoor Attacks and Defenses in Deep Generative Modelscited by 12
  38. 2023FG-Net: Facial Action Unit Detection with Generalizable Pyramidal Featurescited by 12

50 most cited of 251 All 251 (JSON)