# When Scores Learn Geometry: Rate Separations under the Manifold Hypothesis

arXiv: https://arxiv.org/abs/2509.24912 · HTML: https://mlsanity.com/paper/2509.24912/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 28 of its 61 references that are in the map.
- First posted on arXiv 29 Sep 2025.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 28 of 28)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021)
- [Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions](https://mlsanity.com/paper/2209.11215/) (2022)
- [Convergence of denoising diffusion models under the manifold hypothesis](https://mlsanity.com/paper/2208.05314/) (2022)
- [Convergence of score-based generative modeling for general data distributions](https://mlsanity.com/paper/2209.12381/) (2022)
- [Score-Based Generative Models Detect Manifolds](https://mlsanity.com/paper/2206.01018/) (2022)
- [Bayesian imaging using Plug & Play priors: when Langevin meets Tweedie](https://mlsanity.com/paper/2103.04715/) (2021)
- [Solving Linear Inverse Problems Using the Prior Implicit in a Denoiser](https://mlsanity.com/paper/2007.13640/) (2020)
- [Two for One: Diffusion Models and Force Fields for Coarse-Grained Molecular Dynamics](https://mlsanity.com/paper/2302.00600/) (2023)
- [Convergence of Diffusion Models Under the Manifold Hypothesis in High-Dimensions](https://mlsanity.com/paper/2409.18804/) (2024)
- [Deep Generative Models through the Lens of the Manifold Hypothesis: A Survey and New Connections](https://mlsanity.com/paper/2404.02954/) (2024)
- [Manifolds, Random Matrices and Spectral Gaps: The geometric phases of generative diffusion](https://mlsanity.com/paper/2410.05898/) (2024)
- [Divide-and-Conquer Posterior Sampling for Denoising Diffusion Priors](https://mlsanity.com/paper/2403.11407/) (2024)
- [Mathematical analysis of singularities in the diffusion model under the submanifold assumption](https://mlsanity.com/paper/2301.07882/) (2023)
- [Action-Minimization Meets Generative Modeling: Efficient Transition Path Sampling with the Onsager-Machlup Functional](https://mlsanity.com/paper/2504.18506/) (2025)
- [Improving the Euclidean Diffusion Generation of Manifold Data by Mitigating Score Function Singularity](https://mlsanity.com/paper/2505.09922/) (2025)
- [Provable Maximum Entropy Manifold Exploration via Diffusion Models](https://mlsanity.com/paper/2506.15385/) (2025)
- [Resolving Memorization in Empirical Diffusion Model for Manifold Data in High-Dimensional Spaces](https://mlsanity.com/paper/2505.02508/) (2025)
- [Universal Smoothed Score Functions for Generative Modeling](https://mlsanity.com/paper/2303.11669/) (2023)
- [Diffusion models under low-noise regime](https://mlsanity.com/paper/2506.07841/) (2025)
- [MAP Estimation with Denoisers: Convergence Rates and Guarantees](https://mlsanity.com/paper/2507.15397/) (2025)

## Built on by (most cited first, 14 of 14)

- [Diffusion Model for Manifold Data: Score Decomposition, Curvature, and Statistical Complexity](https://mlsanity.com/paper/2603.20645/) (2026)
- [Diffusion Model's Generalization Can Be Characterized by Inductive Biases toward a Data-Dependent Ridge Manifold](https://mlsanity.com/paper/2602.06021/) (2026)
- [Flow Matching from Viewpoint of Proximal Operators](https://mlsanity.com/paper/2602.12683/) (2026)
- [Manifold Generalization Provably Proceeds Memorization in Diffusion Models](https://mlsanity.com/paper/2603.23792/) (2026)
- [Understanding diffusion models requires rethinking (again) generalization](https://mlsanity.com/paper/2605.06077/) (2026)
- [Manifold Aware Denoising Score Matching (MAD)](https://mlsanity.com/paper/2603.02452/) (2026)
- [Diffusion Processes on Implicit Manifolds](https://mlsanity.com/paper/2604.07213/) (2026)
- [Support Before Frequency in Discrete Diffusion](https://mlsanity.com/paper/2605.13999/) (2026)
- [Recovering Weighted Tangent Geometry from a Single-Scale Score Field](https://mlsanity.com/paper/2608.22334/) (2026)
- [Provably Learning Diffusion Models under the Manifold Hypothesis: Collapse and Refine](https://mlsanity.com/paper/2605.20235/) (2026)
- [Not All Prediction Targets Keep Training-Free Diffusion Guidance on the Manifold](https://mlsanity.com/paper/2607.00647/) (2026)
- [Matched Queries for Curvature and Density at Branching Junctions](https://mlsanity.com/paper/2609.01319/) (2026)
- [Beyond Gradient Flow: Identifiability and Recovery from Distribution Snapshots](https://mlsanity.com/paper/2609.35060/) (2026)
- [Geometry-Aware Preference Optimization for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2610.04980/) (2026)

## Abstract (arXiv)

Score-based methods, such as diffusion models and Bayesian inverse problems, are often interpreted as learning the data distribution in the low-noise limit (σ → 0). In this work, we propose an alternative perspective: their success arises from implicitly learning the data manifold rather than the full distribution. Our claim is based on a novel analysis of scores in the small-σ regime that reveals a sharp separation of scales: information about the data manifold is Theta(σ-2) stronger than information about the distribution. We argue that this insight suggests a paradigm shift from the less practical goal of distributional learning to the more attainable task of geometric learning, which provably tolerates O(σ-2) larger errors in score approximation. We illustrate this perspective through three consequences: i) in diffusion models, concentration on data support can be achieved with a score error of o(σ-2), whereas recovering the specific data distribution requires a much stricter o(1) error; ii) more surprisingly, learning the uniform distribution on the manifold-an especially structured and useful object-is also O(σ-2) easier; and iii) in Bayesian inverse problems, the maximum entropy prior is O(σ-2) more robust to score errors than generic priors. Finally, we validate our theoretical findings with preliminary experiments on large-scale models, including Stable Diffusion.