mlsanityData to 7 Oct 2026

Regularization can make diffusion models more efficient

arXiv 2502.09151 ↗First posted 13 Feb 2025cs.LG math.ST stat.MLcited by 2

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion models are one of the key architectures of generative AI. Their main drawback, however, is the computational costs. This study indicates that the concept of sparsity, well known especially in statistics, can provide a pathway to more efficient diffusion pipelines. Our mathematical guarantees prove that sparsity can reduce the input dimension's influence on the computational complexity to that of a much smaller intrinsic dimension of the data. Our empirical findings confirm that inducing sparsity can indeed lead to better samples at a lower cost.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 46 of its 65 references that are in the map. First posted on arXiv 13 Feb 2025.

Builds on

46 of 65 references.

Closest ancestors · 23

Same direction, newest first

  1. 2025Memorization and Regularization in Generative Diffusion Models27 Jan
  2. 2024From optimal score matching to optimal sampling11 Sep
  3. 2024Convergence Analysis of Probability Flow ODE for Score-based Generative Models15 Apr
  4. 2024Accelerating Convergence of Score-Based Diffusion Models, Provably6 Mar
  5. 2024Minimax Optimality of Score-based Diffusion Models: Beyond the Density Lower Bound Assumptions23 Feb
  6. 2024Broadening Target Distributions for Accelerated Diffusion Models via a Novel Analysis Approach21 Feb
  7. 2023Analyzing and Improving the Training Dynamics of Diffusion Models5 Dec
  8. 2023Improved Sample Complexity Bounds for Diffusion Model Training23 Nov
  9. 2023Nearly d-Linear Convergence Bounds for Diffusion Models via Stochastic Localization7 Aug
  10. 2023Towards Faster Non-Asymptotic Convergence for Diffusion-Based Generative Models15 Jun
  11. 2023The probability flow ODE is provably fast19 May
  12. 2023Restoration-Degradation Beyond Linear Diffusions: A Non-Asymptotic Analysis For DDIM-Type Samplers6 Mar
11 more
  1. 2023Score Approximation, Estimation and Distribution Recovery of Diffusion Models on Low-Dimensional Data14 Feb
  2. 2022Improved Analysis of Score-based Generative Modeling: User-Friendly Bounds under Minimal Smoothness Assumptions3 Nov
  3. 2022Convergence of score-based generative modeling for general data distributions26 Sep
  4. 2022Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions22 Sep
  5. 2022Convergence of denoising diffusion models under the manifold hypothesis10 Aug
  6. 2022Convergence for score-based generative modeling with polynomial complexity13 Jun
  7. 2021Diffusion Schr"odinger Bridge with Applications to Score-Based Generative Modeling1 Jun
  8. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  9. 2021Maximum Likelihood Training of Score-Based Diffusion Models22 Jan
  10. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  11. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 23

  1. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  2. 2017Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithmscited by 4,283
  3. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  4. 2022Video Diffusion Modelscited by 2,280
  5. 2022GeoDiff: a Geometric Diffusion Model for Molecular Conformation Generationcited by 511
  6. 2017Nonparametric regression using deep neural networks with ReLU activation functioncited by 431
  7. 2019Sliced Score Matching: A Scalable Approach to Density and Score Estimationcited by 381
  8. 2015Norm-Based Capacity Control in Neural Networkscited by 299
  9. 2017Size-Independent Sample Complexity of Neural Networkscited by 289
  10. 2017Variational Dropout Sparsifies Deep Neural Networkscited by 265
  11. 2020Generative Modeling with Denoising Auto-Encoders and Langevin Samplingcited by 118
  12. 2016Learning the Number of Neurons in Deep Networkscited by 69
11 more
  1. 2024Optimal score estimation via empirical Bayes smoothingcited by 60
  2. 2019Survey of Dropout Methods for Deep Neural Networkscited by 44
  3. 2019Learning Sparse Networks Using Targeted Dropoutcited by 40
  4. 2017Sparse-Input Neural Networks for High-dimensional Nonparametric Regression and Classificationcited by 39
  5. 2020Statistical Guarantees for Regularized Neural Networkscited by 25
  6. 2015Estimation of High-Dimensional Graphical Models Using Regularized Score Matchingcited by 20
  7. 2023Sample Complexity Bounds for Score-Matching: Causal Discovery and Generative Modelingcited by 15
  8. 2020Layer Sparsity in Neural Networkscited by 6
  9. 2024How many samples are needed to train a deep neural network?cited by 5
  10. 2023Extremes in High Dimensions: Methods and Scalable Algorithmscited by 3
  11. 2016Balancing Statistical and Computational Precision: A General Theory and Applications to Sparse Regressioncited by 1

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025Minimax Optimality of the Probability Flow ODE for Diffusion Modelscited by 23
  2. 2025Non-asymptotic error bounds for probability flow ODEs under weak log-concavitycited by 7

Similar, not cited

Close in wording, no citation either way

  1. 2024Denoising diffusion probabilistic models are optimally adaptive to unknown low dimensionality
  2. 2023Diffusion Models for Generative Artificial Intelligence: An Introduction for Applied Mathematicians
  3. 2025Sparse-to-Sparse Training of Diffusion Models
  4. 2024Towards Faster Training of Diffusion Models: An Inspiration of A Consistency Phenomenon
  5. 2025Low-dimensional adaptation of diffusion models: Convergence in total variation
  6. 2025Diffusion Models: A Mathematical Introduction
  7. 2023On the Generalization of Diffusion Model
  8. 2026Diffusion Models Are Statistically Optimal for Learning Low-Dimensional Multi-Modal Distributions