# Score-based generative models learn manifold-like structures with constrained mixing

arXiv: https://arxiv.org/abs/2311.09952 · HTML: https://mlsanity.com/paper/2311.09952/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 4 of its 8 references that are in the map.
- First posted on arXiv 16 Nov 2023.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 4 of 4)

- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Diagnosing and Enhancing VAE Models](https://mlsanity.com/paper/1903.05789/) (2019)
- [Robust and interpretable blind image denoising via bias-free convolutional neural networks](https://mlsanity.com/paper/1906.05478/) (2019)

## Built on by (most cited first, 9 of 9)

- [Diffusion Models as Constrained Samplers for Optimization with Unknown Constraints](https://mlsanity.com/paper/2402.18012/) (2024)
- [Manifolds, Random Matrices and Spectral Gaps: The geometric phases of generative diffusion](https://mlsanity.com/paper/2410.05898/) (2024)
- [The Unreasonable Effectiveness of Gaussian Score Approximation for Diffusion Models and its Applications](https://mlsanity.com/paper/2412.09726/) (2024)
- [Understanding and Mitigating Memorization in Generative Models via Sharpness of Probability Landscapes](https://mlsanity.com/paper/2412.04140/) (2024)
- [How Do Flow Matching Models Memorize and Generalize in Sample Data Subspaces?](https://mlsanity.com/paper/2410.23594/) (2024)
- [Score-based Pullback Riemannian Geometry: Extracting the Data Manifold Geometry using Anisotropic Flows](https://mlsanity.com/paper/2410.01950/) (2024)
- [Be Tangential to Manifold: Discovering Riemannian Metric for Diffusion Models](https://mlsanity.com/paper/2510.05509/) (2025)
- [Exploring Out-of-distribution Detection for Sparse-view Computed Tomography with Diffusion Models](https://mlsanity.com/paper/2411.06308/) (2024)
- [One Inverse Step is a Convex Program: Bayes-Limit Calibration of Diffusion Inversion](https://mlsanity.com/paper/2608.23094/) (2026)

## Abstract (arXiv)

How do score-based generative models (SBMs) learn the data distribution supported on a low-dimensional manifold? We investigate the score model of a trained SBM through its linear approximations and subspaces spanned by local feature vectors. During diffusion as the noise decreases, the local dimensionality increases and becomes more varied between different sample sequences. Importantly, we find that the learned vector field mixes samples by a non-conservative field within the manifold, although it denoises with normal projections as if there is an energy function in off-manifold directions. At each noise level, the subspace spanned by the local features overlap with an effective density function. These observations suggest that SBMs can flexibly mix samples with the learned score field while carefully maintaining a manifold-like structure of the data distribution.