mlsanityData to 14 Oct 2026

Understanding Diffusion Models: A Unified Perspective

arXiv 2208.11970 ↗First posted 25 Aug 2022cs.LG cs.CVcited by 340

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion models have shown incredible capabilities as generative models; indeed, they power the current state-of-the-art models on text-conditioned image generation such as Imagen and DALL-E 2. In this work we review, demystify, and unify the understanding of diffusion models across both variational and score-based perspectives. We first derive Variational Diffusion Models (VDM) as a special case of a Markovian Hierarchical Variational Autoencoder, where three key assumptions enable tractable computation and scalable optimization of the ELBO. We then prove that optimizing a VDM boils down to learning a neural network to predict one of three potential objectives: the original source input from any arbitrary noisification of it, the original source noise from any arbitrarily noisified input, or the score function of a noisified input at any arbitrary noise level. We then dive deeper into what it means to learn the score function, and connect the variational perspective of a diffusion model explicitly with the Score-based Generative Modeling perspective through Tweedie's Formula. Lastly, we cover how to learn a conditional distribution using diffusion models via guidance.

Key facts. Cited by 340 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 16 of its 27 references that are in the map. First posted on arXiv 25 Aug 2022.

Builds on

16 of 27 references.

Closest ancestors · 7

Same direction, newest first

  1. 2022Classifier-Free Diffusion Guidance26 Jul
  2. 2021Variational Diffusion Models1 Jul
  3. 2021Cascaded Diffusion Models for High Fidelity Image Generation30 May
  4. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  6. 2020Denoising Diffusion Probabilistic Models19 Jun
  7. 2020Improved Techniques for Training Score-Based Generative Models16 Jun

Other references · 9

  1. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  2. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  3. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  4. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  5. 2016Improving Variational Inference with Inverse Autoregressive Flowcited by 640
  6. 2019Sliced Score Matching: A Scalable Approach to Density and Score Estimationcited by 384
  7. 2016Ladder Variational Autoencoderscited by 318
  8. 2021How to Train Your Energy-Based Modelscited by 274
  9. 2018Deep Energy Estimator Networkscited by 34

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 11; 2023: 82; 2024: 106; 2025: 97; 2026: 44.

340 papers.

Sort:

By area:

  1. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,114
  2. 2022Is Conditional Generative Modeling all you need for Decision-Making?cited by 516
  3. 2023DreamLLM: Synergistic Multimodal Comprehension and Creationcited by 297
  4. 2023Q-Diffusion: Quantizing Diffusion Modelscited by 273
  5. 2023LLM-grounded Diffusion: Enhancing Prompt Understanding of Text-to-Image Diffusion Models with Large Language Modelscited by 243
  6. 2024Score identity Distillation: Exponentially Fast Distillation of Pretrained Diffusion Models for One-Step Generationcited by 214
  7. 2023Scaling Autoregressive Multi-Modal Models: Pretraining and Instruction Tuningcited by 170
  8. 2023StyleDiffusion: Controllable Disentangled Style Transfer via Diffusion Modelscited by 155
  9. 2023Concept Sliders: LoRA Adaptors for Precise Control in Diffusion Modelscited by 153
  10. 2023Fair Diffusion: Instructing Text-to-Image Generation Models on Fairnesscited by 151
  11. 2022NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360deg Viewscited by 148
  12. 2024Dynamical Regimes of Diffusion Modelscited by 142
38 more
  1. 2022SE(3)-DiffusionFields: Learning smooth cost functions for joint grasp and motion optimization through diffusioncited by 127
  2. 2023AR-Diffusion: Auto-Regressive Diffusion Model for Text Generationcited by 118
  3. 2024GaussCtrl: Multi-View Consistent Text-Driven 3D Gaussian Splatting Editingcited by 115
  4. 2023Diffusion with Forward Models: Solving Stochastic Inverse Problems Without Direct Supervisioncited by 112
  5. 2023HoloDiffusion: Training a 3D Diffusion Model using 2D Imagescited by 106
  6. 2023Kosmos-G: Generating Images in Context with Multimodal Large Language Modelscited by 105
  7. 2023Self-Supervised Learning for Time Series Analysis: Taxonomy, Progress, and Prospectscited by 99
  8. 2022Diffusion Visual Counterfactual Explanationscited by 96
  9. 2024Understanding Hallucinations in Diffusion Models through Mode Interpolationcited by 91
  10. 2023Input Perturbation Reduces Exposure Bias in Diffusion Modelscited by 90
  11. 2025Inference-Time Alignment in Diffusion Models with Reward-Guided Generation: Tutorial and Reviewcited by 88
  12. 2024Controllable Generation with Text-to-Image Diffusion Models: A Surveycited by 83
  13. 2022BeLFusion: Latent Diffusion for Behavior-Driven Human Motion Predictioncited by 82
  14. 2023Robustness of AI-Image Detectors: Fundamental Limits and Practical Attackscited by 81
  15. 2024MM-Interleaved: Interleaved Image-Text Generative Modeling via Multi-modal Feature Synchronizercited by 80
  16. 2024D4M: Dataset Distillation via Disentangled Diffusion Modelcited by 77
  17. 2023Diffusion Action Segmentationcited by 74
  18. 2024An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimizationcited by 73
  19. 2023Diffusion Models for Time Series Applications: A Surveycited by 72
  20. 2024Diffusion Models, Image Super-Resolution And Everything: A Surveycited by 64
  21. 2024Riemannian Flow Matching Policy for Robot Motion Learningcited by 63
  22. 2023On the Design Fundamentals of Diffusion Models: A Surveycited by 60
  23. 2024Score-based Diffusion Models via Stochastic Differential Equations -- a Technical Tutorialcited by 60
  24. 2022Rickrolling the Artist: Injecting Backdoors into Text Encoders for Text-to-Image Synthesiscited by 59
  25. 2023Concept Algebra for (Score-Based) Text-Controlled Generative Modelscited by 59
  26. 2023Consistent Diffusion Models: Mitigating Sampling Drift by Learning to be Consistentcited by 58
  27. 2024Diffusion Models and Representation Learning: A Surveycited by 54
  28. 2023Intriguing properties of generative classifierscited by 52
  29. 2024Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Surveycited by 52
  30. 2024Direct Unlearning Optimization for Robust and Safe Text-to-Image Modelscited by 50
  31. 2024Tutorial on Diffusion Models for Imaging and Visioncited by 49
  32. 2022Compositional Score Modeling for Simulation-based Inferencecited by 48
  33. 2024Flow Matching with Gaussian Process Priors for Probabilistic Time Series Forecastingcited by 43
  34. 2024Efficient Diffusion Models: A Comprehensive Survey from Principles to Practicescited by 41
  35. 2023AtmoRep: A stochastic model of atmosphere dynamics using large scale representation learningcited by 40
  36. 2023Diffusion Model-Augmented Behavioral Cloningcited by 38
  37. 2024Efficient Generation of Targeted and Transferable Adversarial Examples for Vision-Language Models Via Diffusion Modelscited by 38
  38. 2025Dynamic Search for Inference-Time Alignment in Diffusion Modelscited by 37

50 most cited of 340 All 340 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2021A Variational Perspective on Diffusion-Based Generative Models and Score Matching
  2. 2023Neural Diffusion Models
  3. 2023DiffEnc: Variational Diffusion with a Learned Encoder
  4. 2022Let us Build Bridges: Understanding and Extending Diffusion Generative Models
  5. 2025Diffusion Models: A Mathematical Introduction
  6. 2023Variational Diffusion Auto-encoder: Latent Space Extraction from Pre-trained Diffusion Models
  7. 2022How Much is Enough? A Study on Diffusion Times in Score-based Generative Models
  8. 2022Diffusion Models in Vision: A Survey