mlsanityData to 7 Oct 2026

Deep Networks as Denoising Algorithms: Sample-Efficient Learning of Diffusion Models in High-Dimensional Graphical Models

arXiv 2309.11420 ↗First posted 20 Sep 2023cs.LG math.ST stat.MLcited by 39

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

We investigate the approximation efficiency of score functions by deep neural networks in diffusion-based generative modeling. While existing approximation theories utilize the smoothness of score functions, they suffer from the curse of dimensionality for intrinsically high-dimensional data. This limitation is pronounced in graphical models such as Markov random fields, common for image distributions, where the approximation efficiency of score functions remains unestablished. To address this, we observe score functions can often be well-approximated in graphical models through variational inference denoising algorithms. Furthermore, these algorithms are amenable to efficient neural network representation. We demonstrate this in examples of graphical models, including Ising models, conditional Ising models, restricted Boltzmann machines, and sparse encoding models. Combined with off-the-shelf discretization error bounds for diffusion-based sampling, we provide an efficient sample complexity bound for diffusion-based generative modeling when the score function is learned by deep neural networks.

Key facts. Cited by 39 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 52 of its 101 references that are in the map. First posted on arXiv 20 Sep 2023.

Builds on

52 of 101 references.

Closest ancestors · 17

Same direction, newest first

  1. 2023Nearly d-Linear Convergence Bounds for Diffusion Models via Stochastic Localization7 Aug
  2. 2023Learning Mixtures of Gaussians Using the DDPM Objective3 Jul
  3. 2023Towards Faster Non-Asymptotic Convergence for Diffusion-Based Generative Models15 Jun
  4. 2023The probability flow ODE is provably fast19 May
  5. 2023Restoration-Degradation Beyond Linear Diffusions: A Non-Asymptotic Analysis For DDIM-Type Samplers6 Mar
  6. 2023Diffusion Models are Minimax Optimal Distribution Estimators3 Mar
  7. 2023Score Approximation, Estimation and Distribution Recovery of Diffusion Models on Low-Dimensional Data14 Feb
  8. 2022Improved Analysis of Score-based Generative Modeling: User-Friendly Bounds under Minimal Smoothness Assumptions3 Nov
  9. 2022Convergence of score-based generative modeling for general data distributions26 Sep
  10. 2022Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions22 Sep
  11. 2022Diffusion Models in Vision: A Survey10 Sep
  12. 2022Diffusion Models: A Comprehensive Survey of Methods and Applications2 Sep
5 more
  1. 2022Let us Build Bridges: Understanding and Extending Diffusion Generative Models31 Aug
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  3. 2021Structured Denoising Diffusion Models in Discrete State-Spaces7 Jul
  4. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov

Other references · 35

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  3. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  4. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  5. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  6. 2016Variational Inference: A Review for Statisticianscited by 1,466
  7. 2016Error bounds for approximations with deep ReLU networkscited by 567
  8. 2021Grad-TTS: A Diffusion Probabilistic Model for Text-to-Speechcited by 445
  9. 2019Algorithm Unrolling: Interpretable, Efficient Deep Learning for Signal and Image Processingcited by 411
  10. 2014Breaking the Curse of Dimensionality with Convex Neural Networkscited by 329
  11. 2023Transformers as Statisticians: Provable In-Context Learning with In-Context Algorithm Selectioncited by 291
  12. 2022Transformers Learn Shortcuts to Automatacited by 260
23 more
  1. 2023Looped Transformers as Programmable Computerscited by 224
  2. 2015Conditional Random Fields as Recurrent Neural Networkscited by 213
  3. 2013Expectation Propagation for approximate Bayesian inferencecited by 172
  4. 2017Optimal Errors and Phase Transitions in High-Dimensional Generalized Linear Modelscited by 115
  5. 2020A Unified View on Graph Neural Networks as Graph Signal Denoisingcited by 110
  6. 2021Statistically Meaningful Approximation: a Case Study on Approximating Turing Machines with Transformerscited by 101
  7. 2018Theoretical Linear Convergence of Unfolded ISTA and its Practical Weights and Thresholdscited by 83
  8. 2023White-Box Transformers via Sparse Rate Reductioncited by 77
  9. 2016Convolutional Neural Networks Analyzed via Convolutional Sparse Codingcited by 51
  10. 2021A unifying tutorial on Approximate Message Passingcited by 49
  11. 2023Reward-Directed Conditional Diffusion: Provable Distribution Estimation and Reward Improvementcited by 49
  12. 2022Sampling from the Sherrington-Kirkpatrick Gibbs measure via algorithmic stochastic localizationcited by 48
  13. 2019The Barron Space and the Flow-induced Function Spaces for Neural Network Modelscited by 39
  14. 2023Sampling, Diffusions, and Stochastic Localizationcited by 38
  15. 2023Sampling with flows, diffusion and autoregressive neural networks: A spin-glass perspectivecited by 26
  16. 2023Emergence of Segmentation with Minimalistic White-Box Transformerscited by 23
  17. 2022A Proximal Algorithm for Samplingcited by 21
  18. 2021Parametric Complexity Bounds for Approximating PDEs with Neural Networkscited by 18
  19. 2021Local convexity of the TAP free energy and AMP convergence for Z2-synchronizationcited by 15
  20. 2022Neural Network Approximations of PDEs Beyond Linearity: A Representational Perspectivecited by 15
  21. 2018An Instability in Variational Inference for Topic Modelscited by 11
  22. 2021Variational Inference in high-dimensional linear regressioncited by 10
  23. 2022Sudakov-Fernique post-AMP, and a new proof of the local convexity of the TAP free energycited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 23; 2025: 8; 2026: 6.

39 papers.

Sort:

By area:

  1. 2024A Phase Transition in Diffusion Models Reveals the Hierarchical Nature of Datacited by 88
  2. 2023Transformers as Decision Makers: Provable In-Context Reinforcement Learning via Supervised Pretrainingcited by 82
  3. 2024An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimizationcited by 72
  4. 2024Gradient Guidance for Diffusion Models: An Optimization Perspectivecited by 59
  5. 2024Unveil Conditional Diffusion Models with Classifier-free Guidance: A Sharp Statistical Theorycited by 58
  6. 2024Theoretical Insights for Diffusion Guidance: A Case Study for Gaussian Mixture Modelscited by 46
  7. 2024Learning general Gaussian mixtures with efficient score matchingcited by 43
  8. 2024Accelerating Diffusion Models with Parallel Sampling: Inference at Sub-Linear Time Complexitycited by 42
  9. 2024Learning Mixtures of Gaussians Using Diffusion Modelscited by 41
  10. 2024Neural Network-Based Score Estimation in Diffusion Models: Optimization and Generalizationcited by 39
  11. 2023Analysis of learning a flow-based generative model from limited sample complexitycited by 38
  12. 2025Low-dimensional adaptation of diffusion models: Convergence in total variationcited by 38
27 more
  1. 2024Stochastic Runge-Kutta Methods: Provable Acceleration of Diffusion Modelscited by 35
  2. 2024Contractive Diffusion Probabilistic Modelscited by 29
  3. 2024Towards a Theoretical Understanding of the 'Reversal Curse' via Training Dynamicscited by 27
  4. 2024Diffusion Model for Data-Driven Black-Box Optimizationcited by 24
  5. 2025How Compositional Generalization and Creativity Improve as Diffusion Models are Trainedcited by 24
  6. 2025Dimension-Free Convergence of Diffusion Models for Approximate Gaussian Mixturescited by 23
  7. 2025TREAD: Token Routing for Efficient Architecture-agnostic Diffusion Trainingcited by 21
  8. 2024Broadening Target Distributions for Accelerated Diffusion Models via a Novel Analysis Approachcited by 19
  9. 2024Constrained Diffusion Models via Dual Trainingcited by 19
  10. 2024U-Nets as Belief Propagation: Efficient Classification, Denoising, and Diffusion in Generative Hierarchical Modelscited by 17
  11. 2025Sample Complexity and Representation Ability of Test-time Scaling Paradigmscited by 12
  12. 2024Masked Completion via Structured Diffusion with White-Box Transformerscited by 9
  13. 2024Theory on Score-Mismatched Diffusion Models and Zero-Shot Conditional Samplerscited by 6
  14. 2024Diffusion Transformer Captures Spatial-Temporal Dependencies: A Theory for Gaussian Process Datacited by 5
  15. 2025Localized Diffusion Modelscited by 5
  16. 2024Cryptographic Hardness of Score Estimationcited by 3
  17. 2024Neural Networks Generalize on Low Complexity Datacited by 3
  18. 2026Generating from Discrete Distributions Using Diffusions: Insights from Random Constraint Satisfaction Problemscited by 3
  19. 2024Provable Statistical Rates for Consistency Diffusion Modelscited by 2
  20. 2025Computational bottlenecks for denoising diffusionscited by 2
  21. 2024Unrolled denoising networks provably learn optimal Bayesian inferencecited by 1
  22. 2026Multi-Subspace Multi-Modal Modeling for Diffusion Models: Estimation, Convergence and Mixture of Expertscited by 1
  23. 2026Provably Reliable Classifier Guidance via Cross-Entropy Controlcited by 1
  24. 2026Data-Efficient Hierarchical Goal-Conditioned Reinforcement Learning via Normalizing Flowscited by 1
  25. 2026Diffusion Models for Adaptive Sequential Data Generationcited by 1
  26. 2025Diffusion Transformers for Imputation: Statistical Efficiency and Uncertainty Quantificationcited by 0
  27. 2026Sample complexity bounds for categorical Markov random fields via Discrete Diffusionscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Understanding Diffusion Models: A Unified Perspective
  2. 2023To smooth a cloud or to pin it down: Guarantees and Insights on Score Matching in Denoising Diffusion Models
  3. 2024Efficient Denoising using Score Embedding in Score-based Diffusion Models
  4. 2024Analyzing Neural Network-Based Generative Diffusion Models through Convex Optimization
  5. 2025Diffusion Models: A Mathematical Introduction
  6. 2021A Variational Perspective on Diffusion-Based Generative Models and Score Matching
  7. 2026From Score Learning to Discretized Sampling: An End-to-End Generalization Analysis of Diffusion Models
  8. 2023Denoising Diffusion Samplers