mlsanityData to 7 Oct 2026

A Good Score Does not Lead to A Good Generative Model

arXiv 2401.04856 ↗First posted 10 Jan 2024cs.LG stat.MLcited by 36

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Score-based Generative Models (SGMs) is one leading method in generative modeling, renowned for their ability to generate high-quality samples from complex, high-dimensional data distributions. The method enjoys empirical success and is supported by rigorous theoretical convergence properties. In particular, it has been shown that SGMs can generate samples from a distribution that is close to the ground-truth if the underlying score function is learned well, suggesting the success of SGM as a generative model. We provide a counter-example in this paper. Through the sample complexity argument, we provide one specific setting where the score function is learned well. Yet, SGMs in this setting can only output samples that are Gaussian blurring of training data points, mimicking the effects of kernel density estimation. The finding resonates a series of recent finding that reveal that SGMs can demonstrate strong memorization effect and fail to generate.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 33 of its 41 references that are in the map. First posted on arXiv 10 Jan 2024.

Builds on

33 of 41 references.

Closest ancestors · 20

Same direction, newest first

  1. 2023On Memorization in Diffusion Models4 Oct
  2. 2023Generalization in diffusion models arises from geometry-adaptive harmonic representations4 Oct
  3. 2023Nearly d-Linear Convergence Bounds for Diffusion Models via Stochastic Localization7 Aug
  4. 2023Towards Faster Non-Asymptotic Convergence for Diffusion-Based Generative Models15 Jun
  5. 2023Understanding and Mitigating Copying in Diffusion Models31 May
  6. 2023Error Bounds for Flow Matching Methods26 May
  7. 2023The probability flow ODE is provably fast19 May
  8. 2023Restoration-Degradation Beyond Linear Diffusions: A Non-Asymptotic Analysis For DDIM-Type Samplers6 Mar
  9. 2023Diffusion Models are Minimax Optimal Distribution Estimators3 Mar
  10. 2023Score Approximation, Estimation and Distribution Recovery of Diffusion Models on Low-Dimensional Data14 Feb
  11. 2022Score-based Generative Modeling Secretly Minimizes the Wasserstein Distance13 Dec
  12. 2022Improved Analysis of Score-based Generative Modeling: User-Friendly Bounds under Minimal Smoothness Assumptions3 Nov
8 more
  1. 2022Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions22 Sep
  2. 2022Convergence of denoising diffusion models under the manifold hypothesis10 Aug
  3. 2022Convergence for score-based generative modeling with polynomial complexity13 Jun
  4. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  6. 2021Diffusion Schr"odinger Bridge with Applications to Score-Based Generative Modeling1 Jun
  7. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  8. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 13

  1. 2022Flow Matching for Generative Modelingcited by 5,426
  2. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  3. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,176
  4. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,139
  5. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  6. 2023Stochastic Interpolants: A Unifying Framework for Flows and Diffusionscited by 897
  7. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Modelscited by 439
  8. 2018Adversarial Audio Synthesiscited by 222
  9. 2022FastDiff: A Fast Conditional Diffusion Model for High-Quality Speech Synthesiscited by 136
  10. 2020Generative Modeling with Denoising Auto-Encoders and Langevin Samplingcited by 118
  11. 2018IntroVAE: Introspective Variational Autoencoders for Photographic Image Synthesiscited by 85
  12. 2023Analysis of learning a flow-based generative model from limited sample complexitycited by 38
1 more
  1. 2023Statistically Optimal Generative Modeling with Maximum Deviation from the Empirical Distributioncited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 10; 2025: 13; 2026: 13.

36 papers.

Sort:

By area:

  1. 2025Why Diffusion Models Don't Memorize: The Role of Implicit Dynamical Regularization in Trainingcited by 78
  2. 2024Accelerating Convergence of Score-Based Diffusion Models, Provablycited by 74
  3. 2024Optimal score estimation via empirical Bayes smoothingcited by 60
  4. 2025On the Closed-Form of Flow Matching: Generalization Does Not Arise from Target Stochasticitycited by 48
  5. 2024A Geometric Framework for Understanding Memorization in Generative Modelscited by 40
  6. 2025Memorization and Regularization in Generative Diffusion Modelscited by 39
  7. 2024Understanding Generalizability of Diffusion Models Requires Rethinking the Hidden Gaussian Structurecited by 38
  8. 2025Minimax Optimality of the Probability Flow ODE for Diffusion Modelscited by 23
  9. 2024How Do Flow Matching Models Memorize and Generalize in Sample Data Subspaces?cited by 19
  10. 2025Taking a Big Step: Large Learning Rates in Denoising Score Matching Prevent Memorizationcited by 16
  11. 2025On the Interpolation Effect of Score Smoothing in Diffusion Modelscited by 15
  12. 2024Wasserstein proximal operators describe score-based generative models and resolve memorizationcited by 14
24 more
  1. 2024Score-based generative models are provably robust: an uncertainty quantification perspectivecited by 14
  2. 2024Replication in Visual Diffusion Models: A Survey and Outlookcited by 11
  3. 2025Resolving Memorization in Empirical Diffusion Model for Manifold Data in High-Dimensional Spacescited by 8
  4. 2025From Navigation to Refinement: Revealing the Two-Stage Nature of Flow-based Diffusion Models through Oracle Velocitycited by 7
  5. 2026A Random Matrix Theory Perspective on the Consistency of Diffusion Modelscited by 5
  6. 2026Local Coverage Governs Memorization in Diffusion Modelscited by 5
  7. 2025Algorithm- and Data-Dependent Generalization Bounds for Diffusion Modelscited by 4
  8. 2025Origins of Creativity in Attention-Based Diffusion Modelscited by 4
  9. 2026Smoothing the Score Function to Enhance Generalization in Diffusion Modelscited by 3
  10. 2026An exact information theory of generalization phase transitions in Bayesian diffusion modelscited by 3
  11. 2024Localized Schr"odinger Bridge Samplercited by 2
  12. 2024Solving Prior Distribution Mismatch in Diffusion Models via Optimal Transportcited by 2
  13. 2025Generative diffusion models from a PDE perspectivecited by 2
  14. 2025A PDE Perspective on Generative Diffusion Modelscited by 2
  15. 2025On The Hidden Biases of Flow Matching Samplerscited by 2
  16. 2026Understanding diffusion models requires rethinking (again) generalizationcited by 2
  17. 2026Diffusion Models Are Statistically Optimal for Learning Low-Dimensional Multi-Modal Distributionscited by 2
  18. 2026The two clocks and the innovation window: When and how generative models learn rulescited by 1
  19. 2026Where the Score Lives: A Wavelet View of Diffusioncited by 1
  20. 2026Double Descent and Malign Overfitting in Diffusion Modelscited by 1
  21. 2026A Theoretical Analysis of Memory and Overfitting Phenomena in Stochastic Interpolation Modelscited by 0
  22. 2026PAC-DP: PAC-Bayesian Diffusion Policy Learningcited by 0
  23. 2026First-Order Stationarity of Reverse Diffusionscited by 0
  24. 2026First Learn, Then Memorize: The Spectral Bias of Diffusion Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Moderating the Generalization of Score-based Generative Model
  2. 2023Closed-Form Diffusion Models
  3. 2022Score-Based Generative Models Detect Manifolds
  4. 2024An analysis of the noise schedule for score-based generative models
  5. 2021Score-based Generative Modeling in Latent Space
  6. 2022Convergence of score-based generative modeling for general data distributions
  7. 2026Non-asymptotic Convergence of Stochastic Gradient Descent in Score-based Generative Models
  8. 2024Score-based generative models break the curse of dimensionality in learning a family of sub-Gaussian probability distributions