mlsanityData to 7 Oct 2026

Mathematical analysis of singularities in the diffusion model under the submanifold assumption

arXiv 2301.07882 ↗First posted 19 Jan 2023cs.LG cs.NA math.NAcited by 20

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

This paper concerns the mathematical analyses of the diffusion model in machine learning. The drift term of the backward sampling process is represented as a conditional expectation involving the data distribution and the forward diffusion. The training process aims to find such a drift function by minimizing the mean-squared residue related to the conditional expectation. Using small-time approximations of the Green's function of the forward diffusion, we show that the analytical mean drift function in DDPM and the score function in SGM asymptotically blow up in the final stages of the sampling process for singular data distributions such as those concentrated on lower-dimensional manifolds, and are therefore difficult to approximate by a network. To overcome this difficulty, we derive a new target function and associated loss, which remains bounded even for singular data distributions. We validate the theoretical findings with several numerical examples.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 21 of its 34 references that are in the map. First posted on arXiv 19 Jan 2023.

Builds on

21 of 34 references.

Closest ancestors · 11

Same direction, newest first

  1. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models2 Nov
  2. 2022Diffusion Models: A Comprehensive Survey of Methods and Applications2 Sep
  3. 2022Understanding Diffusion Models: A Unified Perspective25 Aug
  4. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps2 Jun
  5. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  6. 2022Fast Sampling of Diffusion Models with Exponential Integrator29 Apr
  7. 2022Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models17 Jan
  8. 2021Score-Based Generative Modeling with Critically-Damped Langevin Diffusion14 Dec
  9. 2021Structured Denoising Diffusion Models in Discrete State-Spaces7 Jul
  10. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  11. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 10

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  2. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  3. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  4. 2018Neural Ordinary Differential Equationscited by 3,733
  5. 2019Normalizing Flows for Probabilistic Modeling and Inferencecited by 1,315
  6. 2020Generative Modeling with Denoising Auto-Encoders and Langevin Samplingcited by 118
  7. 2021Deep Generative Learning via Schr"odinger Bridgecited by 103
  8. 2021Deep Generative Modeling for Protein Designcited by 17
  9. 2021Interpreting diffusion score matching using normalizing flowcited by 9
  10. 2021DeepParticle: learning invariant measure by a deep neural network minimizing Wasserstein distance on data generated from an interacting particle methodcited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 8; 2026: 5.

20 papers.

Sort:

By area:

  1. 2024Deep Generative Models through the Lens of the Manifold Hypothesis: A Survey and New Connectionscited by 47
  2. 2024A Geometric View of Data Complexity: Efficient Local Intrinsic Dimension Estimation with Diffusion Modelscited by 41
  3. 2024A Geometric Framework for Understanding Memorization in Generative Modelscited by 40
  4. 2025Memorization and Regularization in Generative Diffusion Modelscited by 39
  5. 2024A Geometric Explanation of the Likelihood OOD Detection Paradoxcited by 22
  6. 2025Improving the Euclidean Diffusion Generation of Manifold Data by Mitigating Score Function Singularitycited by 14
  7. 2024Tackling the Singularities at the Endpoints of Time Intervals in Diffusion Modelscited by 13
  8. 2025When Scores Learn Geometry: Rate Separations under the Manifold Hypothesiscited by 13
  9. 2024Masked Completion via Structured Diffusion with White-Box Transformerscited by 9
  10. 2025Resolving Memorization in Empirical Diffusion Model for Manifold Data in High-Dimensional Spacescited by 8
  11. 2025Anomaly Detection and Generation with Diffusion Models: A Surveycited by 8
  12. 2025When and how can inexact generative models still sample from the data manifold?cited by 4
8 more
  1. 2025Pathway to O(√d) Complexity bound under Wasserstein metric of flow-based modelscited by 4
  2. 2025Optimal Stopping in Latent Diffusion Modelscited by 1
  3. 2026Preconditioned One-Step Generative Modeling for Bayesian Inverse Problems in Function Spacescited by 1
  4. 2026Intrinsic Wasserstein Rates for Score-Based Generative Models on Smooth Manifoldscited by 1
  5. 2024Inconsistencies In Consistency Models: Better ODE Solving Does Not Imply Better Samplescited by 0
  6. 2026SSI-DM: Singularity Skipping Inversion of Diffusion Modelscited by 0
  7. 2026ML-based approach to classification and generation of structured light propagation in turbulent mediacited by 0
  8. 2026One Inverse Step is a Convex Program: Bayes-Limit Calibration of Diffusion Inversioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Diffusion Models are Minimax Optimal Distribution Estimators
  2. 2024Convergence of Diffusion Models Under the Manifold Hypothesis in High-Dimensions
  3. 2026Diffusion Models Are Statistically Optimal for Learning Low-Dimensional Multi-Modal Distributions
  4. 2024Non-asymptotic bounds for forward processes in denoising diffusions: Ornstein-Uhlenbeck is hard to beat
  5. 2023Fast Diffusion Probabilistic Model Sampling through the lens of Backward Error Analysis
  6. 2023Score-based Diffusion Models in Function Space
  7. 2023Restoration-Degradation Beyond Linear Diffusions: A Non-Asymptotic Analysis For DDIM-Type Samplers
  8. 2025Information-Theoretic Proofs for Diffusion Sampling