On the Anisotropy of Score-Based Generative Models
On the map: Diffusion models → Diffusion foundations and sampling
Abstract (arXiv)
We investigate the role of network architecture in shaping the inductive biases of modern score-based generative models. To this end, we introduce the Score Anisotropy Directions (SADs), architecture-dependent directions that reveal how different networks preferentially capture data structure. Our analysis suggests that SADs form adaptive bases aligned with the architecture's output geometry, providing a principled way to predict generalization ability in score models prior to training. Through both synthetic data and standard image benchmarks, we demonstrate that SADs reliably capture fine-grained model behavior and correlate with downstream performance, as measured by Wasserstein metrics. Our work offers a new lens for explaining and predicting directional biases of generative models.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 24 of its 32 references that are in the map. First posted on arXiv 27 Oct 2025.
Builds on
24 of 32 references.
Closest ancestors · 8
Same direction, newest first
- 2024On Inductive Biases That Enable Generalization of Diffusion Transformers28 Oct
- 2024Detecting, Explaining, and Mitigating Memorization in Diffusion Models31 Jul
- 2023On Memorization in Diffusion Models4 Oct
- 2023Generalization in diffusion models arises from geometry-adaptive harmonic representations4 Oct
- 2022Scalable Diffusion Models with Transformers19 Dec
- 2021Improved Denoising Diffusion Probabilistic Models18 Feb
- 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
- 2020Denoising Diffusion Probabilistic Models19 Jun
Other references · 16
- 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
- 2017Progressive Growing of GANs for Improved Quality, Stability, and Variationcited by 2,967
- 2016Neural Architecture Search with Reinforcement Learningcited by 1,489
- 2018On the Spectral Bias of Neural Networkscited by 1,321
- 2023Extracting Training Data from Diffusion Modelscited by 723
- 2019Deep Double Descent: Where Bigger Models and More Data Hurtcited by 667
- 2020Bayesian Deep Learning and a Probabilistic Perspective of Generalizationcited by 480
- 2022Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Modelscited by 418
- 2017Stochastic Gradient Descent as Approximate Bayesian Inferencecited by 298
- 2020Inductive Biases for Deep Learning of Higher-Level Cognitioncited by 273
- 2019Robust and interpretable blind image denoising via bias-free convolutional neural networkscited by 87
- 2024Understanding Hallucinations in Diffusion Models through Mode Interpolationcited by 87
4 more
- 2020Neural Anisotropy Directionscited by 17
- 2025Towards Understanding Text Hallucination of Diffusion Models via Local Generation Biascited by 11
- 2024Geometric Inductive Biases of Deep Networks: The Role of Data and Architecturecited by 2
- 2025Trustworthy Image Super-Resolution via Generative Pseudoinversecited by 1
Built on by
Citing papers per year: 2025: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2026Where the Score Lives: A Wavelet View of Diffusion
- 2024Moderating the Generalization of Score-based Generative Model
- 2020Improved Techniques for Training Score-Based Generative Models
- 2024An analysis of the noise schedule for score-based generative models
- 2023Score Mismatching for Generative Modeling
- 2022Elucidating the Design Space of Diffusion-Based Generative Models
- 2025Beyond Log-Concavity and Score Regularity: Improved Convergence Bounds for Score-Based Generative Models in W2-distance
- 2022Improved Analysis of Score-based Generative Modeling: User-Friendly Bounds under Minimal Smoothness Assumptions