mlsanityData to 7 Oct 2026

Provable Acceleration for Diffusion Models under Minimal Assumptions

arXiv 2410.23285 ↗First posted 30 Oct 2024cs.LG cs.AI math.OCcited by 27

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Score-based diffusion models, while achieving minimax optimality for sampling, are often hampered by slow sampling speeds due to the high computational burden of score function evaluations. Despite the recent remarkable empirical advances in speeding up the score-based samplers, theoretical understanding of acceleration techniques remains largely limited. To bridge this gap, we propose a novel training-free acceleration scheme for stochastic samplers. Under minimal assumptions -- namely, L2-accurate score estimates and a finite second-moment condition on the target distribution -- our accelerated sampler provably achieves varepsilon-accuracy in total variation within widetildeO(d5/4/√varepsilon) iterations, thereby significantly improving upon the widetildeO(d/varepsilon) iteration complexity of standard score-based samplers for varepsilon≤ 1/√d. Notably, our convergence theory does not rely on restrictive assumptions on the target distribution or higher-order score estimation guarantees.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 58 of its 75 references that are in the map. First posted on arXiv 30 Oct 2024.

Builds on

58 of 75 references.

Closest ancestors · 40

Same direction, newest first

  1. 2024Denoising diffusion probabilistic models are optimally adaptive to unknown low dimensionality24 Oct
  2. 2024Instance-dependent Convergence Theory for Diffusion Models17 Oct
  3. 2024Stochastic Runge-Kutta Methods: Provable Acceleration of Diffusion Models7 Oct
  4. 2024O(d/T) Convergence Theory for Diffusion Probabilistic Models under Minimal Assumptions27 Sep
  5. 2024Convergence of Diffusion Models Under the Manifold Hypothesis in High-Dimensions27 Sep
  6. 2024From optimal score matching to optimal sampling11 Sep
  7. 2024Breaking the Curse of Dimensionality: Diffusion Models Efficiently Learn Low-Dimensional Distributions4 Sep
  8. 2024A Sharp Convergence Theory for The Probability Flow ODEs of Diffusion Models5 Aug
  9. 2024Reverse Transition Kernel: A Flexible Framework to Accelerate Diffusion Inference26 May
  10. 2024Accelerating Diffusion Models with Parallel Sampling: Inference at Sub-Linear Time Complexity24 May
  11. 2024Adapting to Unknown Low-Dimensional Structures in Score-Based Diffusion Models23 May
  12. 2024Convergence Analysis of Probability Flow ODE for Score-based Generative Models15 Apr
28 more
  1. 2024An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimization11 Apr
  2. 2024Accelerating Convergence of Score-Based Diffusion Models, Provably6 Mar
  3. 2024Minimax Optimality of Score-based Diffusion Models: Beyond the Density Lower Bound Assumptions23 Feb
  4. 2024Score-based Diffusion Models via Stochastic Differential Equations -- a Technical Tutorial12 Feb
  5. 2024Convergence Analysis for General Probability Flow ODEs of Diffusion Models in Wasserstein Distances31 Jan
  6. 2024Neural Network-Based Score Estimation in Diffusion Models: Optimization and Generalization28 Jan
  7. 2023SA-Solver: Stochastic Adams Solver for Fast Sampling of Diffusion Models10 Sep
  8. 2023Towards Faster Non-Asymptotic Convergence for Diffusion-Based Generative Models15 Jun
  9. 2023The probability flow ODE is provably fast19 May
  10. 2023Restoration-Degradation Beyond Linear Diffusions: A Non-Asymptotic Analysis For DDIM-Type Samplers6 Mar
  11. 2023Diffusion Models are Minimax Optimal Distribution Estimators3 Mar
  12. 2023Consistency Models2 Mar
  13. 2023Score Approximation, Estimation and Distribution Recovery of Diffusion Models on Low-Dimensional Data14 Feb
  14. 2022Improved Analysis of Score-based Generative Modeling: User-Friendly Bounds under Minimal Smoothness Assumptions3 Nov
  15. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models2 Nov
  16. 2022Convergence of score-based generative modeling for general data distributions26 Sep
  17. 2022Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions22 Sep
  18. 2022Let us Build Bridges: Understanding and Extending Diffusion Generative Models31 Aug
  19. 2022Convergence of denoising diffusion models under the manifold hypothesis10 Aug
  20. 2022Convergence for score-based generative modeling with polynomial complexity13 Jun
  21. 2022Score-Based Generative Models Detect Manifolds2 Jun
  22. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps2 Jun
  23. 2022Fast Sampling of Diffusion Models with Exponential Integrator29 Apr
  24. 2021Structured Denoising Diffusion Models in Discrete State-Spaces7 Jul
  25. 2021Diffusion Schr"odinger Bridge with Applications to Score-Based Generative Modeling1 Jun
  26. 2021Gotta Go Fast When Generating Data with Score-Based Models28 May
  27. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  28. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov

Other references · 18

  1. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  2. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  3. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  4. 2023Stochastic Interpolants: A Unifying Framework for Flows and Diffusionscited by 897
  5. 2021Solving Inverse Problems in Medical Imaging with Score-Based Generative Modelscited by 514
  6. 2019Sliced Score Matching: A Scalable Approach to Density and Score Estimationcited by 381
  7. 2021Score-based diffusion models for accelerated MRIcited by 308
  8. 2023IDQL: Implicit Q-Learning as an Actor-Critic Method with Diffusion Policiescited by 307
  9. 2023Imitating Human Behaviour with Diffusion Modelscited by 260
  10. 2022Offline Reinforcement Learning via High-Fidelity Generative Behavior Modelingcited by 177
  11. 2023Protein Design with Guided Discrete Diffusioncited by 154
  12. 2020Generative Modeling with Denoising Auto-Encoders and Langevin Samplingcited by 118
6 more
  1. 2024Optimal score estimation via empirical Bayes smoothingcited by 60
  2. 2022Normalizing flow neural networks by JKO schemecited by 39
  3. 2023Sampling, Diffusions, and Stochastic Localizationcited by 38
  4. 2023Analysis of learning a flow-based generative model from limited sample complexitycited by 38
  5. 2023Convergence of flow-based generative models via proximal gradient descent in Wasserstein spacecited by 34
  6. 2024Posterior sampling via Langevin dynamics based on generative priorscited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 18; 2026: 9.

27 papers.

Sort:

By area:

  1. 2025Low-dimensional adaptation of diffusion models: Convergence in total variationcited by 38
  2. 2025Minimax Optimality of the Probability Flow ODE for Diffusion Modelscited by 23
  3. 2025Dimension-Free Convergence of Diffusion Models for Approximate Gaussian Mixturescited by 23
  4. 2025Breaking AR's Sampling Bottleneck: Provable Acceleration via Diffusion Language Modelscited by 23
  5. 2025Advancing Wasserstein Convergence Analysis of Score-Based Models: Insights from Discretization and Second-Order Accelerationcited by 17
  6. 2026Confidence-Based Decoding is Provably Efficient for Diffusion Language Modelscited by 17
  7. 2025Faster Diffusion Models via Higher-Order Approximationcited by 16
  8. 2025Optimal Convergence Analysis of DDPM for General Distributionscited by 16
  9. 2025Wasserstein Convergence of Score-based Generative Models under Semiconvexity and Discontinuous Gradientscited by 15
  10. 2026High-accuracy sampling for diffusion models and log-concave distributionscited by 13
  11. 2026Adaptation to Intrinsic Dependence in Diffusion Language Modelscited by 9
  12. 2025Provable Efficiency of Guidance in Diffusion Models for General Data Distributioncited by 8
15 more
  1. 2025Angle Domain Guidance: Latent Diffusion Requires Rotation Rather Than Extrapolationcited by 8
  2. 2025Sublinear iterations can suffice even for DDPMscited by 7
  3. 2025Towards a unified framework for guided diffusion modelscited by 7
  4. 2025Information-Theoretic Proofs for Diffusion Samplingcited by 6
  5. 2025Almost Linear Convergence under Minimal Score Assumptions: Quantized Transition Diffusioncited by 6
  6. 2025Improving Diffusion-based Inverse Algorithms under Few-Step Constraint via Learnable Linear Extrapolationcited by 4
  7. 2026Denoising growth complexity: Data geometry and certified schedules for diffusion samplingcited by 2
  8. 2026ConvergeFlow: Language Flow with Provable Convergence to Token Embeddingscited by 2
  9. 2025Non-asymptotic convergence bound of conditional diffusion modelscited by 1
  10. 2026Load--Reserve Wasserstein Propagation for Isotropic Diffusion Samplerscited by 1
  11. 2026Diffusion Models for Adaptive Sequential Data Generationcited by 1
  12. 2026Diffusion Models Adapt to Low-Dimensional Structure Under Flexible Coefficient Choicescited by 1
  13. 2025Capturing Conditional Dependence via Auto-regressive Diffusion Modelscited by 0
  14. 2025Are First-Order Diffusion Samplers Really Slower? A Fast Forward-Value Approachcited by 0
  15. 2026Training Parallel Speculative Draft Models by Directly Minimizing Expected Decoding Roundscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Broadening Target Distributions for Accelerated Diffusion Models via a Novel Analysis Approach
  2. 2024Unified Convergence Analysis for Score-Based Diffusion Models with Deterministic Samplers
  3. 2026Efficient Sampling with Discrete Diffusion Models: Sharp and Adaptive Guarantees
  4. 2026Diffusion Path Samplers via Sequential Monte Carlo
  5. 2026Query Lower Bounds for Diffusion Sampling
  6. 2026Diffusion Models with Heavy-Tailed Targets: Score Estimation and Sampling Guarantees
  7. 2024Convergence of Score-Based Discrete Diffusion Models: A Discrete-Time Analysis
  8. 2021Maximum Likelihood Training of Score-Based Diffusion Models