mlsanityData to 14 Oct 2026

A Variational Perspective on Diffusion-Based Generative Models and Score Matching

arXiv 2106.02808 ↗First posted 5 Jun 2021cs.LGcited by 183

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Discrete-time diffusion-based generative models and score matching methods have shown promising results in modeling high-dimensional image data. Recently, Song et al. (2021) show that diffusion processes that transform data into noise can be reversed via learning the score function, i.e. the gradient of the log-density of the perturbed data. They propose to plug the learned score function into an inverse formula to define a generative diffusion process. Despite the empirical success, a theoretical underpinning of this procedure is still lacking. In this work, we approach the (continuous-time) generative diffusion directly and derive a variational framework for likelihood estimation, which includes continuous-time normalizing flows as a special case, and can be seen as an infinitely deep variational autoencoder. Under this framework, we show that minimizing the score-matching loss is equivalent to maximizing a lower bound of the likelihood of the plug-in reverse SDE proposed by Song et al. (2021), bridging the theoretical gap.

Key facts. Cited by 183 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 26 of its 61 references that are in the map. First posted on arXiv 5 Jun 2021.

Builds on

26 of 61 references.

Closest ancestors · 4

Same direction, newest first

  1. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  2. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  3. 2020Denoising Diffusion Probabilistic Models19 Jun
  4. 2020Improved Techniques for Training Score-Based Generative Models16 Jun

Other references · 22

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  4. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  5. 2013Auto-Encoding Variational Bayescited by 4,053
  6. 2018Neural Ordinary Differential Equationscited by 3,779
  7. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  8. 2016Density estimation using Real NVPcited by 2,042
  9. 2021Image Super-Resolution via Iterative Refinementcited by 1,511
  10. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,354
  11. 2018FFJORD: Free-form Continuous Dynamics for Scalable Reversible Generative Modelscited by 594
  12. 20213D Shape Generation and Completion through Point-Voxel Diffusioncited by 516
10 more
  1. 2019Sliced Score Matching: A Scalable Approach to Density and Score Estimationcited by 384
  2. 2021How to Train Your Energy-Based Modelscited by 274
  3. 2020Scalable Gradients for Stochastic Differential Equationscited by 271
  4. 2020Learning Gradient Fields for Shape Generationcited by 245
  5. 2018Neural Autoregressive Flowscited by 208
  6. 2019Neural Stochastic Differential Equations: Deep Latent Gaussian Models in the Diffusion Limitcited by 176
  7. 2020SurVAE Flows: Surjections to Bridge the Gap between VAEs and Flowscited by 71
  8. 2021Infinitely Deep Bayesian Neural Networks with Stochastic Differential Equationscited by 38
  9. 2017Variational Walkback: Learning a Transition Operator as a Stochastic Recurrent Netcited by 30
  10. 2019A RAD approach to deep mixture modelscited by 18

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 12; 2022: 35; 2023: 57; 2024: 41; 2025: 24; 2026: 14.

183 papers.

Sort:

By area:

  1. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 3,045
  2. 2021Variational Diffusion Modelscited by 1,153
  3. 2022Diffusion Models: A Comprehensive Survey of Methods and Applicationscited by 1,114
  4. 2022Diffusion Models in Vision: A Surveycited by 1,015
  5. 2021Score-based Generative Modeling in Latent Spacecited by 636
  6. 2021Tackling the Generative Learning Trilemma with Denoising Diffusion GANscited by 516
  7. 2021Come-Closer-Diffuse-Faster: Accelerating Conditional Diffusion Models for Inverse Problems through Stochastic Contractioncited by 307
  8. 2024Generative Flows on Discrete State-Spaces: Enabling Multimodal Flows with Applications to Protein Co-Designcited by 305
  9. 2023Understanding Diffusion Objectives as the ELBO with Simple Data Augmentationcited by 304
  10. 2021SegDiff: Image Segmentation with Diffusion Probabilistic Modelscited by 280
  11. 2022Convergence of denoising diffusion models under the manifold hypothesiscited by 242
  12. 2022A Survey on Generative Diffusion Modelcited by 240
38 more
  1. 2021Score-Based Generative Modeling with Critically-Damped Langevin Diffusioncited by 233
  2. 2025DiffusionNFT: Online Diffusion Reinforcement with Forward Processcited by 230
  3. 2021Likelihood Training of Schr"odinger Bridge using Forward-Backward SDEs Theorycited by 219
  4. 2022Riemannian Score-Based Generative Modellingcited by 219
  5. 2023SE(3) diffusion model with application to protein backbone generationcited by 219
  6. 2021Autoregressive Diffusion Modelscited by 210
  7. 2022DiffRF: Rendering-Guided 3D Radiance Field Diffusioncited by 204
  8. 2022Speech Enhancement and Dereverberation with Diffusion-based Generative Modelscited by 197
  9. 2022Learning Fast Samplers for Diffusion Models by Differentiating Through Sample Qualitycited by 182
  10. 2022Generative Modelling With Inverse Heat Dissipationcited by 167
  11. 2023Flow Matching in Latent Spacecited by 160
  12. 2023Diffusion Schr"odinger Bridge Matchingcited by 153
  13. 2023StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Modelscited by 152
  14. 2022An optimal control perspective on diffusion-based generative modelingcited by 147
  15. 2022GENIE: Higher-Order Denoising Diffusion Solverscited by 127
  16. 2022Score-Based Generative Models Detect Manifoldscited by 122
  17. 2022Riemannian Diffusion Modelscited by 122
  18. 2022Wavelet Diffusion Models are fast and scalable Image Generatorscited by 117
  19. 2023Improved sampling via learned diffusionscited by 104
  20. 2022Let us Build Bridges: Understanding and Extending Diffusion Generative Modelscited by 103
  21. 2021Soft Truncation: A Universal Training Technique of Score-based Diffusion Model for High Precision Score Estimationcited by 97
  22. 2024PnP-Flow: Plug-and-Play Image Restoration with Flow Matchingcited by 97
  23. 2024Align Your Steps: Optimizing Sampling Schedules in Diffusion Modelscited by 96
  24. 2023Reflected Diffusion Modelscited by 87
  25. 2022Maximum Likelihood Training for Score-Based Diffusion ODEs by High-Order Denoising Score Matchingcited by 80
  26. 2022Diffusion Models for Counterfactual Explanationscited by 79
  27. 2023Diffusion Models for Black-Box Optimizationcited by 76
  28. 2023Dirichlet Diffusion Score Model for Biological Sequence Generationcited by 75
  29. 2022DOLCE: A Model-Based Probabilistic Diffusion Framework for Limited-Angle CT Reconstructioncited by 74
  30. 2024An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimizationcited by 73
  31. 2022Unifying Diffusion Models' Latent Space, with Applications to CycleDiffusion and Guidancecited by 67
  32. 2024A Survey on Diffusion Models for Time Series and Spatio-Temporal Datacited by 67
  33. 2022The Schr"odinger Bridge between Gaussian Measures has a Closed Formcited by 63
  34. 2023Diffusion Models for Constrained Domainscited by 63
  35. 2022Improved Vector Quantized Diffusion Modelscited by 62
  36. 2023On the Design Fundamentals of Diffusion Models: A Surveycited by 60
  37. 2024Score-based Diffusion Models via Stochastic Differential Equations -- a Technical Tutorialcited by 60
  38. 2023Improved Techniques for Maximum Likelihood Estimation for Diffusion ODEscited by 54

50 most cited of 183 All 183 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2022Understanding Diffusion Models: A Unified Perspective
  2. 2026A Unified Measure-Theoretic View of Diffusion, Score-Based, and Flow Matching Generative Models
  3. 2026A Stochastic Analysis Perspective on Diffusion Models
  4. 2024Discrete vs. Continuous Trade-offs for Generative Models
  5. 2025Adaptivity and Convergence of Probability Flow ODEs in Diffusion Generative Models
  6. 2021Conditional Image Generation with Score-Based Diffusion Models
  7. 2022From Denoising Diffusions to Denoising Markov Models
  8. 2026Where the Score Lives: A Wavelet View of Diffusion