# Mathematical analysis of singularities in the diffusion model under the submanifold assumption arXiv: https://arxiv.org/abs/2301.07882 · HTML: https://mlsanity.com/paper/2301.07882/ ## Key facts - Cited by 20 papers in the map (data through 14 Oct 2026). - Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. - Builds on 21 of its 34 references that are in the map. - First posted on arXiv 19 Jan 2023. ## Placed in - Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/) ## Builds on (most cited first, 21 of 21) - [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020) - [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015) - [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020) - [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015) - [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019) - [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018) - [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022) - [DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps](https://mlsanity.com/paper/2206.00927/) (2022) - [Structured Denoising Diffusion Models in Discrete State-Spaces](https://mlsanity.com/paper/2107.03006/) (2021) - [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019) - [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022) - [DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models](https://mlsanity.com/paper/2211.01095/) (2022) - [Fast Sampling of Diffusion Models with Exponential Integrator](https://mlsanity.com/paper/2204.13902/) (2022) - [Understanding Diffusion Models: A Unified Perspective](https://mlsanity.com/paper/2208.11970/) (2022) - [Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.06503/) (2022) - [Score-Based Generative Modeling with Critically-Damped Langevin Diffusion](https://mlsanity.com/paper/2112.07068/) (2021) - [Generative Modeling with Denoising Auto-Encoders and Langevin Sampling](https://mlsanity.com/paper/2002.00107/) (2020) - [Deep Generative Learning via Schr"odinger Bridge](https://mlsanity.com/paper/2106.10410/) (2021) - [Deep Generative Modeling for Protein Design](https://mlsanity.com/paper/2109.13754/) (2021) - [Interpreting diffusion score matching using normalizing flow](https://mlsanity.com/paper/2107.10072/) (2021) - [DeepParticle: learning invariant measure by a deep neural network minimizing Wasserstein distance on data generated from an interacting particle method](https://mlsanity.com/paper/2111.01356/) (2021) ## Built on by (most cited first, 20 of 20) - [Deep Generative Models through the Lens of the Manifold Hypothesis: A Survey and New Connections](https://mlsanity.com/paper/2404.02954/) (2024) - [A Geometric View of Data Complexity: Efficient Local Intrinsic Dimension Estimation with Diffusion Models](https://mlsanity.com/paper/2406.03537/) (2024) - [A Geometric Framework for Understanding Memorization in Generative Models](https://mlsanity.com/paper/2411.00113/) (2024) - [Memorization and Regularization in Generative Diffusion Models](https://mlsanity.com/paper/2501.15785/) (2025) - [A Geometric Explanation of the Likelihood OOD Detection Paradox](https://mlsanity.com/paper/2403.18910/) (2024) - [Improving the Euclidean Diffusion Generation of Manifold Data by Mitigating Score Function Singularity](https://mlsanity.com/paper/2505.09922/) (2025) - [When Scores Learn Geometry: Rate Separations under the Manifold Hypothesis](https://mlsanity.com/paper/2509.24912/) (2025) - [Tackling the Singularities at the Endpoints of Time Intervals in Diffusion Models](https://mlsanity.com/paper/2403.08381/) (2024) - [Masked Completion via Structured Diffusion with White-Box Transformers](https://mlsanity.com/paper/2404.02446/) (2024) - [Resolving Memorization in Empirical Diffusion Model for Manifold Data in High-Dimensional Spaces](https://mlsanity.com/paper/2505.02508/) (2025) - [Anomaly Detection and Generation with Diffusion Models: A Survey](https://mlsanity.com/paper/2506.09368/) (2025) - [When and how can inexact generative models still sample from the data manifold?](https://mlsanity.com/paper/2508.07581/) (2025) - [Pathway to O(√d) Complexity bound under Wasserstein metric of flow-based models](https://mlsanity.com/paper/2512.06702/) (2025) - [Optimal Stopping in Latent Diffusion Models](https://mlsanity.com/paper/2510.08409/) (2025) - [Intrinsic Wasserstein Rates for Score-Based Generative Models on Smooth Manifolds](https://mlsanity.com/paper/2605.15822/) (2026) - [Preconditioned One-Step Generative Modeling for Bayesian Inverse Problems in Function Spaces](https://mlsanity.com/paper/2603.14798/) (2026) - [Inconsistencies In Consistency Models: Better ODE Solving Does Not Imply Better Samples](https://mlsanity.com/paper/2411.08954/) (2024) - [SSI-DM: Singularity Skipping Inversion of Diffusion Models](https://mlsanity.com/paper/2602.02193/) (2026) - [ML-based approach to classification and generation of structured light propagation in turbulent media](https://mlsanity.com/paper/2604.14208/) (2026) - [One Inverse Step is a Convex Program: Bayes-Limit Calibration of Diffusion Inversion](https://mlsanity.com/paper/2608.23094/) (2026) ## Abstract (arXiv) This paper concerns the mathematical analyses of the diffusion model in machine learning. The drift term of the backward sampling process is represented as a conditional expectation involving the data distribution and the forward diffusion. The training process aims to find such a drift function by minimizing the mean-squared residue related to the conditional expectation. Using small-time approximations of the Green's function of the forward diffusion, we show that the analytical mean drift function in DDPM and the score function in SGM asymptotically blow up in the final stages of the sampling process for singular data distributions such as those concentrated on lower-dimensional manifolds, and are therefore difficult to approximate by a network. To overcome this difficulty, we derive a new target function and associated loss, which remains bounded even for singular data distributions. We validate the theoretical findings with several numerical examples.