# Diffusion Model's Generalization Can Be Characterized by Inductive Biases toward a Data-Dependent Ridge Manifold

arXiv: https://arxiv.org/abs/2602.06021 · HTML: https://mlsanity.com/paper/2602.06021/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 15 of its 38 references that are in the map.
- First posted on arXiv 5 Feb 2026.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 15 of 15)

- [Generalization in diffusion models arises from geometry-adaptive harmonic representations](https://mlsanity.com/paper/2310.02557/) (2023)
- [An analytic theory of creativity in convolutional diffusion models](https://mlsanity.com/paper/2412.20292/) (2024)
- [Why Diffusion Models Don't Memorize: The Role of Implicit Dynamical Regularization in Training](https://mlsanity.com/paper/2505.17638/) (2025)
- [KL Convergence Guarantees for Score diffusion models under minimal data assumptions](https://mlsanity.com/paper/2308.12240/) (2023)
- [On the Closed-Form of Flow Matching: Generalization Does Not Arise from Target Stochasticity](https://mlsanity.com/paper/2506.03719/) (2025)
- [Memorization and Regularization in Generative Diffusion Models](https://mlsanity.com/paper/2501.15785/) (2025)
- [Generalization through variance: how noise shapes inductive biases in diffusion models](https://mlsanity.com/paper/2504.12532/) (2025)
- [The Emergence of Reproducibility and Generalizability in Diffusion Models](https://mlsanity.com/paper/2310.05264/) (2023)
- [Denoising Score Matching with Random Features: Insights on Diffusion Models from Precise Learning Curves](https://mlsanity.com/paper/2502.00336/) (2025)
- [Diffusion Models and the Manifold Hypothesis: Log-Domain Smoothing is Geometry Adaptive](https://mlsanity.com/paper/2510.02305/) (2025)
- [Taking a Big Step: Large Learning Rates in Denoising Score Matching Prevent Memorization](https://mlsanity.com/paper/2502.03435/) (2025)
- [Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusion](https://mlsanity.com/paper/2502.21278/) (2025)
- [On the Interpolation Effect of Score Smoothing in Diffusion Models](https://mlsanity.com/paper/2502.19499/) (2025)
- [When Scores Learn Geometry: Rate Separations under the Manifold Hypothesis](https://mlsanity.com/paper/2509.24912/) (2025)
- [Provable Separations between Memorization and Generalization in Diffusion Models](https://mlsanity.com/paper/2511.03202/) (2025)

## Built on by (most cited first, 5 of 5)

- [Making Reconstruction FID Predictive of Diffusion Generation FID](https://mlsanity.com/paper/2603.05630/) (2026)
- [Benign Overfitting Does Not Occur in Diffusion Models](https://mlsanity.com/paper/2607.02671/) (2026)
- [An exact information theory of generalization phase transitions in Bayesian diffusion models](https://mlsanity.com/paper/2607.08041/) (2026)
- [Toward a mechanistic understanding of inference in visual cortex and diffusion models](https://mlsanity.com/paper/2607.15693/) (2026)
- [Scaling and Distilling Text Embeddings for Better Diffusibility](https://mlsanity.com/paper/2610.01016/) (2026)

## Abstract (arXiv)

We study a data-dependent notion of diffusion-model generalization: when a model does not memorize the training set, where do its generated samples go relative to the geometry induced by the data? To answer this, we introduce a time-dependent family of log-density ridge manifolds constructed from the smoothed empirical distribution, and use it to characterize reverse-time inference. Our main result shows that generated samples evolve by a reach-align-slide mechanism: they first enter a neighborhood of the ridge, then their distance to the ridge is controlled by the normal component of training error, and finally their motion along the ridge is controlled by the tangential component. We further connect this geometric picture to training dynamics through directional decompositions of the learned error, and make this link explicit for random feature models, where architectural bias and optimization error can be separated quantitatively. Experiments on synthetic multimodal data and MNIST latent diffusion support the predicted geometric behavior in both low and high dimensions.