mlsanityData to 7 Oct 2026

A Geometric Perspective on Diffusion Models

arXiv 2305.19947 ↗First posted 31 May 2023cs.CV cs.LG stat.MLcited by 17

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Recent years have witnessed significant progress in developing effective training and fast sampling techniques for diffusion models. A remarkable advancement is the use of stochastic differential equations (SDEs) and their marginal-preserving ordinary differential equations (ODEs) to describe data perturbation and generative modeling in a unified framework. In this paper, we carefully inspect the ODE-based sampling of a popular variance-exploding SDE and reveal several intriguing structures of its sampling dynamics. We discover that the data distribution and the noise distribution are smoothly connected with a quasi-linear sampling trajectory and another implicit denoising trajectory that even converges faster. Meanwhile, the denoising trajectory governs the curvature of the corresponding sampling trajectory and its finite differences yield various second-order samplers used in practice. Furthermore, we establish a theoretical relationship between the optimal ODE-based sampling and the classic mean-shift (mode-seeking) algorithm, with which we can characterize the asymptotic behavior of diffusion models and identify the empirical score deviation. Code is available at urlhttps://github.com/zju-pi/diff-sampler.

Key facts. Cited by 17 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 39 of its 73 references that are in the map. First posted on arXiv 31 May 2023.

Builds on

39 of 73 references.

Closest ancestors · 18

Same direction, newest first

  1. 2023TRACT: Denoising Diffusion Models with Transitive Closure Time-Distillation7 Mar
  2. 2023Consistency Models2 Mar
  3. 2023Stable Target Field for Reduced Variance Score Estimation in Diffusion Models1 Feb
  4. 2022Fast Sampling of Diffusion Models via Operator Learning24 Nov
  5. 2022GENIE: Higher-Order Denoising Diffusion Solvers11 Oct
  6. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps2 Jun
  7. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  8. 2022Fast Sampling of Diffusion Models with Exponential Integrator29 Apr
  9. 2022Pseudo Numerical Methods for Diffusion Models on Manifolds20 Feb
  10. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  11. 2021Variational Diffusion Models1 Jul
  12. 2021Score-based Generative Modeling in Latent Space10 Jun
6 more
  1. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  2. 2021Maximum Likelihood Training of Score-Based Diffusion Models22 Jan
  3. 2021Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed7 Jan
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  5. 2020Denoising Diffusion Implicit Models6 Oct
  6. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 21

  1. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  2. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  3. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  4. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  5. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  6. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  7. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  8. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  9. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  10. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  11. 2022Video Diffusion Modelscited by 2,280
  12. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
9 more
  1. 2015LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loopcited by 1,350
  2. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  3. 2019StarGAN v2: Diverse Image Synthesis for Multiple Domainscited by 1,095
  4. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoiserscited by 921
  5. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 620
  6. 2022StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasetscited by 489
  7. 2013Generalized Denoising Auto-Encoders as Generative Modelscited by 113
  8. 2019Neural Empirical Bayescited by 46
  9. 2015A review of mean-shift algorithms for clusteringcited by 18

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 5; 2024: 6; 2025: 4; 2026: 2.

17 papers.

Sort:

By area:

  1. 2024MACE: Mass Concept Erasure in Diffusion Modelscited by 298
  2. 2023Fast ODE-based Sampling for Diffusion Models in Around 5 Stepscited by 83
  3. 2023On the Design Fundamentals of Diffusion Models: A Surveycited by 59
  4. 2024On the Trajectory Regularity of ODE-based Diffusion Samplingcited by 51
  5. 2024Simple and Fast Distillation of Diffusion Modelscited by 31
  6. 2024Conditional Image Synthesis with Diffusion Models: A Surveycited by 29
  7. 2023Adaptive Multi-Teacher Knowledge Distillation with Meta-Learningcited by 15
  8. 2025One-Step Offline Distillation of Diffusion-based Models via Koopman Modelingcited by 9
  9. 2025Adaptive Stochastic Coefficients for Accelerating Diffusion Samplingcited by 6
  10. 2023Your Student is Better Than Expected: Adaptive Teacher-Student Collaboration for Text-Conditional Diffusion Modelscited by 5
  11. 2024Diffusion Features to Bridge Domain Gap for Semantic Segmentationcited by 5
  12. 2025Geometric Regularity in Deterministic Sampling Dynamics of Diffusion-based Generative Modelscited by 3
5 more
  1. 2023Customizing Synthetic Data for Data-Free Student Learningcited by 2
  2. 2026Meltdown: Circuits and Bifurcations in Point-Cloud-Conditioned 3D Diffusion Transformerscited by 1
  3. 2024Trading Devil Final: Backdoor attack via Stock market and Bayesian Optimizationcited by 0
  4. 2025Mean-Shift Distillation for Diffusion Mode Seekingcited by 0
  5. 2026Analyzing and Improving Fast Sampling of Text-to-Image Diffusion Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026A Tutorial on Diffusion Theory: From Differential Equations to Diffusion Models
  2. 2025Diffusion Models under Alternative Noise: Simplified Analysis and Sensitivity
  3. 2022An optimal control perspective on diffusion-based generative modeling
  4. 2024New algorithms for sampling and diffusion models
  5. 2023SEEDS: Exponential SDE Solvers for Fast High-Quality Sampling from Diffusion Models
  6. 2023Restoration-Degradation Beyond Linear Diffusions: A Non-Asymptotic Analysis For DDIM-Type Samplers
  7. 2026High-accuracy and dimension-free sampling with diffusions
  8. 2023Elucidating the solution space of extended reverse-time SDE for diffusion models