mlsanityData to 14 Oct 2026

A solvable generative model with a linear, one-step denoiser

arXiv 2411.17807 ↗First posted 26 Nov 2024cs.LG cs.CVcited by 2

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

We develop an analytically tractable single-step diffusion model based on a linear denoiser and present an explicit formula for the Kullback-Leibler divergence between the generated and sampling distribution, taken to be isotropic Gaussian, showing the effect of finite diffusion time and noise scale. Our study further reveals that the monotonic fall phase of Kullback-Leibler divergence begins when the training dataset size reaches the dimension of the data points. Finally, for large-scale practical diffusion models, we explain why a higher number of diffusion steps enhances production quality based on the theoretical arguments presented before.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 58 of its 75 references that are in the map. First posted on arXiv 26 Nov 2024.

Builds on

58 of 75 references.

Closest ancestors · 24

Same direction, newest first

  1. 2024What does guidance do? A fine-grained analysis in a simple setting19 Sep
  2. 2024Classifier-Free Guidance is a Predictor-Corrector16 Aug
  3. 2024Theoretical Insights for Diffusion Guidance: A Case Study for Gaussian Mixture Models3 Mar
  4. 2024Critical windows: non-asymptotic theory for feature emergence in diffusion models3 Mar
  5. 2023Compositional Abilities Emerge Multiplicatively: Exploring Diffusion Models on a Synthetic Task13 Oct
  6. 2023The Emergence of Reproducibility and Generalizability in Diffusion Models8 Oct
  7. 2023Generalization in diffusion models arises from geometry-adaptive harmonic representations4 Oct
  8. 2023Nearly d-Linear Convergence Bounds for Diffusion Models via Stochastic Localization7 Aug
  9. 2023Learning Mixtures of Gaussians Using the DDPM Objective3 Jul
  10. 2023On the Importance of Noise Scheduling for Diffusion Models26 Jan
  11. 2022Scalable Diffusion Models with Transformers19 Dec
  12. 2022Improved Analysis of Score-based Generative Modeling: User-Friendly Bounds under Minimal Smoothness Assumptions3 Nov
12 more
  1. 2022Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions22 Sep
  2. 2022Classifier-Free Diffusion Guidance26 Jul
  3. 2022Convergence for score-based generative modeling with polynomial complexity13 Jun
  4. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  6. 2021Diffusion Normalizing Flow14 Oct
  7. 2021A Variational Perspective on Diffusion-Based Generative Models and Score Matching5 Jun
  8. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  9. 2021Improved Denoising Diffusion Probabilistic Models18 Feb
  10. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  11. 2020Denoising Diffusion Implicit Models6 Oct
  12. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 34

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  3. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  4. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  6. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  7. 2018Neural Tangent Kernel: Convergence and Generalization in Neural Networkscited by 2,621
  8. 2019Wide Neural Networks of Any Depth Evolve as Linear Models Under Gradient Descentcited by 738
  9. 2023Extracting Training Data from Diffusion Modelscited by 732
  10. 2019Surprises in High-Dimensional Ridgeless Least Squares Interpolationcited by 513
  11. 2022MaxViT: Multi-Axis Vision Transformercited by 476
  12. 2018A Mean Field View of the Landscape of Two-Layers Neural Networkscited by 466
22 more
  1. 2021Explaining Neural Scaling Lawscited by 363
  2. 2017High-dimensional dynamics of generalization error in neural networkscited by 241
  3. 2020Spectrum Dependent Learning Curves in Kernel Regression and Wide Neural Networkscited by 200
  4. 2022Scalable Adaptive Computation for Iterative Generationcited by 195
  5. 2020Spectral Bias and Task-Model Alignment Explain Generalization in Kernel Regression and Infinitely Wide Neural Networkscited by 166
  6. 2015High-Dimensional Asymptotics of Prediction: Ridge Regression and Classificationcited by 148
  7. 2021Learning curves of generic features maps for realistic datasets with a teacher-student modelcited by 126
  8. 2022A Solvable Model of Neural Scaling Lawscited by 116
  9. 2021Generalization error of random features and kernel methods: hypercontractivity and kernel matrix concentrationcited by 110
  10. 2022Self-Consistent Dynamical Field Theory of Kernel Evolution in Wide Neural Networkscited by 105
  11. 2020The Neural Tangent Kernel in High Dimensions: Triple Descent and a Multi-Scale Theory of Generalizationcited by 101
  12. 2020Double Trouble in Double Descent : Bias and Variance(s) in the Lazy Regimecited by 95
  13. 2017A Random Matrix Approach to Neural Networkscited by 80
  14. 2020Solving Linear Inverse Problems Using the Prior Implicit in a Denoisercited by 79
  15. 2020Triple descent and the two kinds of overfitting: Where & why do they appear?cited by 65
  16. 2024Scaling and renormalization in high-dimensional regressioncited by 50
  17. 2023High-dimensional analysis of double descent for linear regression with random projectionscited by 47
  18. 2023Dynamics of Finite Width Kernel and Prediction Fluctuations in Mean Field Neural Networkscited by 43
  19. 2019More Data Can Hurt for Linear Regression: Sample-wise Double Descentcited by 37
  20. 2024Emergence of Hidden Capabilities: Exploring Learning Dynamics in Concept Spacecited by 35
  21. 2022The Onset of Variance-Limited Behavior for Networks in the Lazy and Rich Regimescited by 25
  22. 2023Learning curves for deep structured Gaussian feature modelscited by 13

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Steering Dynamical Regimes of Diffusion Models by Breaking Detailed Balancecited by 3
  2. 2026Diffusion Models Memorize in Training -- and Generalize in Inferencecited by 2

Similar, not cited

Close in wording, no citation either way

  1. 2026A Gaussian Perspective for Distributional Discrepancy in Generative Diffusion Models
  2. 2026A Unified Kullback--Leibler Divergence Analysis of Generative Diffusion Models via Entropy Production Rate
  3. 2025Diffusion Models: A Mathematical Introduction
  4. 2023Towards Faster Non-Asymptotic Convergence for Diffusion-Based Generative Models
  5. 2025Improving Diffusion Generative Models via Truncated Karhunen--Lo`eve Expansion
  6. 2025A Sharp KL-Convergence Analysis for Diffusion Models under Minimal Assumptions
  7. 2025Fewer Denoising Steps or Cheaper Per-Step Inference: Towards Compute-Optimal Diffusion Model Deployment
  8. 2024GUD: Generation with Unified Diffusion