# Denoising Score Matching in Diffusion Models: The Effect of Rank Constraints on Generalization

arXiv: https://arxiv.org/abs/2610.20850 · HTML: https://mlsanity.com/paper/2610.20850/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- First posted on arXiv 14 Oct 2026.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Abstract (arXiv)

Architectural biases restrict the class of functions that a diffusion score network can represent, but their effect on generalization remains poorly understood. We study denoising score matching with score networks having a rank constrained non-linear term and a linear skip connection. Numerical experiments with trainable three-layer networks on data supported on an embedded sphere show that increasing rank lowers training error, but its effect on test error depends on sample size and data dimension. To explain this behavior, we analyze a random-feature score model under rank constraint with Gaussian data supported on a low-dimensional subspace. We characterize the optimal rank-constrained estimator through a truncated singular value decomposition and derive asymptotically precise limiting training and test errors when sample size (n), feature width (p), data dimension (D), and rank (r) scale proportionally with the ambient dimension (d). The resulting spectral characterization distinguishes eigenmodes that improve the empirical fit from those that improve population accuracy. It also identifies the dimension of the orthogonal complement of the data subspace as a natural rank threshold. The theoretical learning curves qualitatively reproduce the observed rank-dependent overfitting, providing a tractable account of how bottlenecks or rank constraints can regularize score estimation and why their effectiveness depends on data dimension and sample size.