mlsanityData to 14 Oct 2026

Improving the Euclidean Diffusion Generation of Manifold Data by Mitigating Score Function Singularity

arXiv 2505.09922 ↗First posted 15 May 2025cs.LGcited by 17

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Euclidean diffusion models have achieved remarkable success in generative modeling across diverse domains, and they have been extended to manifold cases in recent advances. Instead of explicitly utilizing the structure of special manifolds as studied in previous works, in this paper we investigate direct sampling of the Euclidean diffusion models for general manifold-structured data. We reveal the multiscale singularity of the score function in the ambient space, which hinders the accuracy of diffusion-generated samples. We then present an elaborate theoretical analysis of the singularity structure of the score function by decomposing it along the tangential and normal directions of the manifold. To mitigate the singularity and improve the sampling accuracy, we propose two novel methods: (1) Niso-DM, which reduces the scale discrepancies in the score function by utilizing a non-isotropic noise, and (2) Tango-DM, which trains only the tangential component of the score function using a tangential-only loss function. Numerical experiments demonstrate that our methods achieve superior performance on distributions over various manifolds with complex geometries.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 34 of its 51 references that are in the map. First posted on arXiv 15 May 2025.

Builds on

34 of 51 references.

Closest ancestors · 20

Same direction, newest first

  1. 2025Riemannian Denoising Diffusion Probabilistic Models7 May
  2. 2025Analysis of Diffusion Models for Manifold Data1 Feb
  3. 2024Manifolds, Random Matrices and Spectral Gaps: The geometric phases of generative diffusion8 Oct
  4. 2024Breaking the Curse of Dimensionality: Diffusion Models Efficiently Learn Low-Dimensional Distributions4 Sep
  5. 2024Deep Generative Models through the Lens of the Manifold Hypothesis: A Survey and New Connections3 Apr
  6. 2024On gauge freedom, conservativity and intrinsic dimensionality estimation in diffusion models6 Feb
  7. 2023Scaling Riemannian Diffusion Models30 Oct
  8. 2023Generalization in diffusion models arises from geometry-adaptive harmonic representations4 Oct
  9. 2023Score Approximation, Estimation and Distribution Recovery of Diffusion Models on Low-Dimensional Data14 Feb
  10. 2023Mathematical analysis of singularities in the diffusion model under the submanifold assumption19 Jan
  11. 2022Diffusion Models: A Comprehensive Survey of Methods and Applications2 Sep
  12. 2022Riemannian Diffusion Models16 Aug
8 more
  1. 2022Convergence of denoising diffusion models under the manifold hypothesis10 Aug
  2. 2022Score-Based Generative Models Detect Manifolds2 Jun
  3. 2022Riemannian Score-Based Generative Modelling6 Feb
  4. 2021Score-Based Generative Modeling with Critically-Damped Langevin Diffusion14 Dec
  5. 2021Cascaded Diffusion Models for High Fidelity Image Generation30 May
  6. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  7. 2020Denoising Diffusion Probabilistic Models19 Jun
  8. 2020Improved Techniques for Training Score-Based Generative Models16 Jun

Other references · 14

  1. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  2. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  3. 2022Equivariant Diffusion for Molecule Generation in 3Dcited by 677
  4. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 622
  5. 20213D Shape Generation and Completion through Point-Voxel Diffusioncited by 516
  6. 2021SRDiff: Single Image Super-Resolution with Diffusion Probabilistic Modelscited by 415
  7. 2023Flow Matching on General Geometriescited by 265
  8. 2021Neural Descriptor Fields: SE(3)-Equivariant Object Representations for Manipulationcited by 209
  9. 2020Riemannian Continuous Normalizing Flowscited by 145
  10. 2021Moser Flow: Divergence-based Generative Modeling on Manifoldscited by 67
  11. 2022Diagnosing and Fixing Manifold Overfitting in Deep Generative Modelscited by 28
  12. 2023Generative Modeling on Manifolds Through Mixture of Riemannian Diffusion Processescited by 21
2 more
  1. 2024Trivialized Momentum Facilitates Diffusion Generative Modeling on Lie Groupscited by 19
  2. 2021Density estimation on low-dimensional manifolds: an inflation-deflation approachcited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 14.

17 papers.

Sort:

By area:

  1. 2025When Scores Learn Geometry: Rate Separations under the Manifold Hypothesiscited by 14
  2. 2026Flow Matching from Viewpoint of Proximal Operatorscited by 2
  3. 2026Manifold Generalization Provably Proceeds Memorization in Diffusion Modelscited by 2
  4. 2026A Deep Generative Approach to Stratified Learningcited by 2
  5. 2026Intrinsic Wasserstein Rates for Score-Based Generative Models on Smooth Manifoldscited by 2
  6. 2026Recovering Weighted Tangent Geometry from a Single-Scale Score Fieldcited by 1
  7. 2025MAD: Manifold Attracted Diffusioncited by 0
  8. 2025Generative modeling of conditional probability distributions on the level-sets of collective variablescited by 0
  9. 2026SSI-DM: Singularity Skipping Inversion of Diffusion Modelscited by 0
  10. 2026Capturing the Topological Phase Transition and Thermodynamics of the 2D XY Model via Manifold-Aware Score-Based Generative Modelingcited by 0
  11. 2026Score-Guided Proximal Projection: A Unified Geometric Framework for Rectified Flow Editingcited by 0
  12. 2026Efficient Diffusion Models under Nonconvex Equality and Inequality constraints via Landingcited by 0
5 more
  1. 2026FES-FM: Free Energy Surface Sampling via Reduced Flow Matchingcited by 0
  2. 2026Provably Learning Diffusion Models under the Manifold Hypothesis: Collapse and Refinecited by 0
  3. 2026Not All Prediction Targets Keep Training-Free Diffusion Guidance on the Manifoldcited by 0
  4. 2026MaDeL: Manifold-Decomposed Feature Losses for Generative Modelingcited by 0
  5. 2026Geometry-Aware Preference Optimization for Text-to-Image Diffusion Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Diffusion Model for Manifold Data: Score Decomposition, Curvature, and Statistical Complexity
  2. 2025Generative Learning of Densities on Manifolds
  3. 2024Convergence of Diffusion Models Under the Manifold Hypothesis in High-Dimensions
  4. 2025Be Tangential to Manifold: Discovering Riemannian Metric for Diffusion Models
  5. 2025What's Inside Your Diffusion Model? A Score-Based Riemannian Metric to Explore the Data Manifold
  6. 2026Sampling from multi-modal distributions on Riemannian manifolds with training-free stochastic interpolants
  7. 2023Manifold Diffusion Fields
  8. 2026Riemannian Diffusion Models on General Manifolds via Physics-Informed Neural Networks