# Improving the Euclidean Diffusion Generation of Manifold Data by Mitigating Score Function Singularity

arXiv: https://arxiv.org/abs/2505.09922 · HTML: https://mlsanity.com/paper/2505.09922/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 34 of its 51 references that are in the map.
- First posted on arXiv 15 May 2025.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 34)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Cascaded Diffusion Models for High Fidelity Image Generation](https://mlsanity.com/paper/2106.15282/) (2021)
- [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022)
- [Improved Techniques for Training Score-Based Generative Models](https://mlsanity.com/paper/2006.09011/) (2020)
- [Equivariant Diffusion for Molecule Generation in 3D](https://mlsanity.com/paper/2203.17003/) (2022)
- [WaveGrad: Estimating Gradients for Waveform Generation](https://mlsanity.com/paper/2009.00713/) (2020)
- [3D Shape Generation and Completion through Point-Voxel Diffusion](https://mlsanity.com/paper/2104.03670/) (2021)
- [SRDiff: Single Image Super-Resolution with Diffusion Probabilistic Models](https://mlsanity.com/paper/2104.14951/) (2021)
- [Flow Matching on General Geometries](https://mlsanity.com/paper/2302.03660/) (2023)
- [Convergence of denoising diffusion models under the manifold hypothesis](https://mlsanity.com/paper/2208.05314/) (2022)
- [Score-Based Generative Modeling with Critically-Damped Langevin Diffusion](https://mlsanity.com/paper/2112.07068/) (2021)
- [Riemannian Score-Based Generative Modelling](https://mlsanity.com/paper/2202.02763/) (2022)
- [Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulation](https://mlsanity.com/paper/2112.05124/) (2021)
- [Generalization in diffusion models arises from geometry-adaptive harmonic representations](https://mlsanity.com/paper/2310.02557/) (2023)
- [Score Approximation, Estimation and Distribution Recovery of Diffusion Models on Low-Dimensional Data](https://mlsanity.com/paper/2302.07194/) (2023)
- [Riemannian Continuous Normalizing Flows](https://mlsanity.com/paper/2006.10605/) (2020)
- [Score-Based Generative Models Detect Manifolds](https://mlsanity.com/paper/2206.01018/) (2022)
- [Riemannian Diffusion Models](https://mlsanity.com/paper/2208.07949/) (2022)
- [Moser Flow: Divergence-based Generative Modeling on Manifolds](https://mlsanity.com/paper/2108.08052/) (2021)
- [Breaking the Curse of Dimensionality: Diffusion Models Efficiently Learn Low-Dimensional Distributions](https://mlsanity.com/paper/2409.02426/) (2024)
- [Deep Generative Models through the Lens of the Manifold Hypothesis: A Survey and New Connections](https://mlsanity.com/paper/2404.02954/) (2024)
- [Manifolds, Random Matrices and Spectral Gaps: The geometric phases of generative diffusion](https://mlsanity.com/paper/2410.05898/) (2024)
- [Diagnosing and Fixing Manifold Overfitting in Deep Generative Models](https://mlsanity.com/paper/2204.07172/) (2022)
- [On gauge freedom, conservativity and intrinsic dimensionality estimation in diffusion models](https://mlsanity.com/paper/2402.03845/) (2024)
- [Generative Modeling on Manifolds Through Mixture of Riemannian Diffusion Processes](https://mlsanity.com/paper/2310.07216/) (2023)
- [Mathematical analysis of singularities in the diffusion model under the submanifold assumption](https://mlsanity.com/paper/2301.07882/) (2023)
- [Trivialized Momentum Facilitates Diffusion Generative Modeling on Lie Groups](https://mlsanity.com/paper/2405.16381/) (2024)

## Built on by (most cited first, 17 of 17)

- [When Scores Learn Geometry: Rate Separations under the Manifold Hypothesis](https://mlsanity.com/paper/2509.24912/) (2025)
- [Flow Matching from Viewpoint of Proximal Operators](https://mlsanity.com/paper/2602.12683/) (2026)
- [Manifold Generalization Provably Proceeds Memorization in Diffusion Models](https://mlsanity.com/paper/2603.23792/) (2026)
- [A Deep Generative Approach to Stratified Learning](https://mlsanity.com/paper/2604.10650/) (2026)
- [Intrinsic Wasserstein Rates for Score-Based Generative Models on Smooth Manifolds](https://mlsanity.com/paper/2605.15822/) (2026)
- [Recovering Weighted Tangent Geometry from a Single-Scale Score Field](https://mlsanity.com/paper/2608.22334/) (2026)
- [MAD: Manifold Attracted Diffusion](https://mlsanity.com/paper/2509.24710/) (2025)
- [Generative modeling of conditional probability distributions on the level-sets of collective variables](https://mlsanity.com/paper/2512.17374/) (2025)
- [SSI-DM: Singularity Skipping Inversion of Diffusion Models](https://mlsanity.com/paper/2602.02193/) (2026)
- [Capturing the Topological Phase Transition and Thermodynamics of the 2D XY Model via Manifold-Aware Score-Based Generative Modeling](https://mlsanity.com/paper/2602.07548/) (2026)
- [Score-Guided Proximal Projection: A Unified Geometric Framework for Rectified Flow Editing](https://mlsanity.com/paper/2603.05761/) (2026)
- [Efficient Diffusion Models under Nonconvex Equality and Inequality constraints via Landing](https://mlsanity.com/paper/2604.17838/) (2026)
- [FES-FM: Free Energy Surface Sampling via Reduced Flow Matching](https://mlsanity.com/paper/2605.00337/) (2026)
- [Provably Learning Diffusion Models under the Manifold Hypothesis: Collapse and Refine](https://mlsanity.com/paper/2605.20235/) (2026)
- [Not All Prediction Targets Keep Training-Free Diffusion Guidance on the Manifold](https://mlsanity.com/paper/2607.00647/) (2026)
- [MaDeL: Manifold-Decomposed Feature Losses for Generative Modeling](https://mlsanity.com/paper/2610.04419/) (2026)
- [Geometry-Aware Preference Optimization for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2610.04980/) (2026)

## Abstract (arXiv)

Euclidean diffusion models have achieved remarkable success in generative modeling across diverse domains, and they have been extended to manifold cases in recent advances. Instead of explicitly utilizing the structure of special manifolds as studied in previous works, in this paper we investigate direct sampling of the Euclidean diffusion models for general manifold-structured data. We reveal the multiscale singularity of the score function in the ambient space, which hinders the accuracy of diffusion-generated samples. We then present an elaborate theoretical analysis of the singularity structure of the score function by decomposing it along the tangential and normal directions of the manifold. To mitigate the singularity and improve the sampling accuracy, we propose two novel methods: (1) Niso-DM, which reduces the scale discrepancies in the score function by utilizing a non-isotropic noise, and (2) Tango-DM, which trains only the tangential component of the score function using a tangential-only loss function. Numerical experiments demonstrate that our methods achieve superior performance on distributions over various manifolds with complex geometries.