# Generalization in diffusion models arises from geometry-adaptive harmonic representations

arXiv: https://arxiv.org/abs/2310.02557 · HTML: https://mlsanity.com/paper/2310.02557/

## Key facts

- Cited by 185 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 18 of its 40 references that are in the map.
- First posted on arXiv 4 Oct 2023.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 18 of 18)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [Extracting Training Data from Diffusion Models](https://mlsanity.com/paper/2301.13188/) (2023)
- [Maximum Likelihood Training of Score-Based Diffusion Models](https://mlsanity.com/paper/2101.09258/) (2021)
- [Bayesian Deep Learning and a Probabilistic Perspective of Generalization](https://mlsanity.com/paper/2002.08791/) (2020)
- [Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Models](https://mlsanity.com/paper/2212.03860/) (2022)
- [The Little Engine that Could: Regularization by Denoising (RED)](https://mlsanity.com/paper/1611.02862/) (2016)
- [Inductive Biases for Deep Learning of Higher-Level Cognition](https://mlsanity.com/paper/2011.15091/) (2020)
- [Robust and interpretable blind image denoising via bias-free convolutional neural networks](https://mlsanity.com/paper/1906.05478/) (2019)
- [Solving Linear Inverse Problems Using the Prior Implicit in a Denoiser](https://mlsanity.com/paper/2007.13640/) (2020)
- [Learning multi-scale local conditional probability models of images](https://mlsanity.com/paper/2303.02984/) (2023)
- [Investigating Data Memorization in 3D Latent Diffusion Models for Medical Image Synthesis](https://mlsanity.com/paper/2307.01148/) (2023)

## Built on by (most cited first, 30 of 185)

- [Dynamical Regimes of Diffusion Models](https://mlsanity.com/paper/2402.18491/) (2024)
- [An analytic theory of creativity in convolutional diffusion models](https://mlsanity.com/paper/2412.20292/) (2024)
- [Why Diffusion Models Don't Memorize: The Role of Implicit Dynamical Regularization in Training](https://mlsanity.com/paper/2505.17638/) (2025)
- [A Survey on Diffusion Models for Time Series and Spatio-Temporal Data](https://mlsanity.com/paper/2404.18886/) (2024)
- [A Sharp Convergence Theory for The Probability Flow ODEs of Diffusion Models](https://mlsanity.com/paper/2408.02320/) (2024)
- [Breaking the Curse of Dimensionality: Diffusion Models Efficiently Learn Low-Dimensional Distributions](https://mlsanity.com/paper/2409.02426/) (2024)
- [ConceptAttention: Diffusion Transformers Learn Highly Interpretable Features](https://mlsanity.com/paper/2502.04320/) (2025)
- [On the Closed-Form of Flow Matching: Generalization Does Not Arise from Target Stochasticity](https://mlsanity.com/paper/2506.03719/) (2025)
- [DiP: Taming Diffusion Models in Pixel Space](https://mlsanity.com/paper/2511.18822/) (2025)
- [Closed-Form Diffusion Models](https://mlsanity.com/paper/2310.12395/) (2023)
- [Linear Convergence of Diffusion Models Under the Manifold Hypothesis](https://mlsanity.com/paper/2410.09046/) (2024)
- [Denoising diffusion probabilistic models are optimally adaptive to unknown low dimensionality](https://mlsanity.com/paper/2410.18784/) (2024)
- [Towards a Mechanistic Explanation of Diffusion Model Generalization](https://mlsanity.com/paper/2411.19339/) (2024)
- [A Geometric Framework for Understanding Memorization in Generative Models](https://mlsanity.com/paper/2411.00113/) (2024)
- [Understanding Generalizability of Diffusion Models Requires Rethinking the Hidden Gaussian Structure](https://mlsanity.com/paper/2410.24060/) (2024)
- [A Good Score Does not Lead to A Good Generative Model](https://mlsanity.com/paper/2401.04856/) (2024)
- [Sequence-Augmented SE(3)-Flow Matching For Conditional Protein Backbone Generation](https://mlsanity.com/paper/2405.20313/) (2024)
- [Memorization to Generalization: Emergence of Diffusion Models from Associative Memory](https://mlsanity.com/paper/2505.21777/) (2025)
- [Denoising: A Powerful Building-Block for Imaging, Inverse Problems, and Machine Learning](https://mlsanity.com/paper/2409.06219/) (2024)
- [Simple ReFlow: Improved Techniques for Fast Flow Models](https://mlsanity.com/paper/2410.07815/) (2024)
- [The Stronger the Diffusion Model, the Easier the Backdoor: Data Poisoning to Induce Copyright Breaches Without Adjusting Finetuning Pipeline](https://mlsanity.com/paper/2401.04136/) (2024)
- [Influence Functions for Scalable Data Attribution in Diffusion Models](https://mlsanity.com/paper/2410.13850/) (2024)
- [Manifolds, Random Matrices and Spectral Gaps: The geometric phases of generative diffusion](https://mlsanity.com/paper/2410.05898/) (2024)
- [The Unreasonable Effectiveness of Gaussian Score Approximation for Diffusion Models and its Applications](https://mlsanity.com/paper/2412.09726/) (2024)
- [Losing dimensions: Geometric memorization in generative diffusion](https://mlsanity.com/paper/2410.08727/) (2024)
- [There Will Be a Scientific Theory of Deep Learning](https://mlsanity.com/paper/2604.21691/) (2026)
- [Unifying Generation and Prediction on Graphs with Latent Graph Diffusion](https://mlsanity.com/paper/2402.02518/) (2024)
- [How Compositional Generalization and Creativity Improve as Diffusion Models are Trained](https://mlsanity.com/paper/2502.12089/) (2025)
- [Bigger Isn't Always Memorizing: Early Stopping Overparameterized Diffusion Models](https://mlsanity.com/paper/2505.16959/) (2025)
- [Locality in Image Diffusion Models Emerges from Data Statistics](https://mlsanity.com/paper/2509.09672/) (2025)

## Abstract (arXiv)

Deep neural networks (DNNs) trained for image denoising are able to generate high-quality samples with score-based reverse diffusion algorithms. These impressive capabilities seem to imply an escape from the curse of dimensionality, but recent reports of memorization of the training set raise the question of whether these networks are learning the "true" continuous density of the data. Here, we show that two DNNs trained on non-overlapping subsets of a dataset learn nearly the same score function, and thus the same density, when the number of training images is large enough. In this regime of strong generalization, diffusion-generated images are distinct from the training set, and are of high visual quality, suggesting that the inductive biases of the DNNs are well-aligned with the data density. We analyze the learned denoising functions and show that the inductive biases give rise to a shrinkage operation in a basis adapted to the underlying image. Examination of these bases reveals oscillating harmonic structures along contours and in homogeneous regions. We demonstrate that trained denoisers are inductively biased towards these geometry-adaptive harmonic bases since they arise not only when the network is trained on photographic images, but also when it is trained on image classes supported on low-dimensional manifolds for which the harmonic basis is suboptimal. Finally, we show that when trained on regular image classes for which the optimal basis is known to be geometry-adaptive and harmonic, the denoising performance of the networks is near-optimal.