# Faster Diffusion Models via Higher-Order Approximation

arXiv: https://arxiv.org/abs/2506.24042 · HTML: https://mlsanity.com/paper/2506.24042/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 49 of its 55 references that are in the map.
- First posted on arXiv 30 Jun 2025.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 49)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps](https://mlsanity.com/paper/2206.00927/) (2022)
- [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023)
- [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022)
- [Diffusion Models in Vision: A Survey](https://mlsanity.com/paper/2209.04747/) (2022)
- [DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models](https://mlsanity.com/paper/2211.01095/) (2022)
- [On Distillation of Guided Diffusion Models](https://mlsanity.com/paper/2210.03142/) (2022)
- [Fast Sampling of Diffusion Models with Exponential Integrator](https://mlsanity.com/paper/2204.13902/) (2022)
- [UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Models](https://mlsanity.com/paper/2302.04867/) (2023)
- [Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions](https://mlsanity.com/paper/2209.11215/) (2022)
- [Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed](https://mlsanity.com/paper/2101.02388/) (2021)
- [Nearly d-Linear Convergence Bounds for Diffusion Models via Stochastic Localization](https://mlsanity.com/paper/2308.03686/) (2023)
- [Gotta Go Fast When Generating Data with Score-Based Models](https://mlsanity.com/paper/2105.14080/) (2021)
- [Improved Analysis of Score-based Generative Modeling: User-Friendly Bounds under Minimal Smoothness Assumptions](https://mlsanity.com/paper/2211.01916/) (2022)
- [Convergence of score-based generative modeling for general data distributions](https://mlsanity.com/paper/2209.12381/) (2022)
- [DPM-Solver-v3: Improved Diffusion ODE Solver with Empirical Model Statistics](https://mlsanity.com/paper/2310.13268/) (2023)
- [The probability flow ODE is provably fast](https://mlsanity.com/paper/2305.11798/) (2023)
- [Parallel Sampling of Diffusion Models](https://mlsanity.com/paper/2305.16317/) (2023)
- [Let us Build Bridges: Understanding and Extending Diffusion Generative Models](https://mlsanity.com/paper/2208.14699/) (2022)
- [Towards Faster Non-Asymptotic Convergence for Diffusion-Based Generative Models](https://mlsanity.com/paper/2306.09251/) (2023)
- [Accelerating Convergence of Score-Based Diffusion Models, Provably](https://mlsanity.com/paper/2403.03852/) (2024)
- [Restoration-Degradation Beyond Linear Diffusions: A Non-Asymptotic Analysis For DDIM-Type Samplers](https://mlsanity.com/paper/2303.03384/) (2023)
- [SA-Solver: Stochastic Adams Solver for Fast Sampling of Diffusion Models](https://mlsanity.com/paper/2309.05019/) (2023)
- [A Sharp Convergence Theory for The Probability Flow ODEs of Diffusion Models](https://mlsanity.com/paper/2408.02320/) (2024)
- [Convergence of Diffusion Models Under the Manifold Hypothesis in High-Dimensions](https://mlsanity.com/paper/2409.18804/) (2024)

## Built on by (most cited first, 16 of 16)

- [Confidence-Based Decoding is Provably Efficient for Diffusion Language Models](https://mlsanity.com/paper/2603.22248/) (2026)
- [Optimal Convergence Analysis of DDPM for General Distributions](https://mlsanity.com/paper/2510.27562/) (2025)
- [High-accuracy sampling for diffusion models and log-concave distributions](https://mlsanity.com/paper/2602.01338/) (2026)
- [Adaptation to Intrinsic Dependence in Diffusion Language Models](https://mlsanity.com/paper/2602.20126/) (2026)
- [Towards a unified framework for guided diffusion models](https://mlsanity.com/paper/2512.04985/) (2025)
- [High-accuracy and dimension-free sampling with diffusions](https://mlsanity.com/paper/2601.10708/) (2026)
- [Dimension-free error estimate for diffusion model and optimal scheduling](https://mlsanity.com/paper/2512.01820/) (2025)
- [Simultaneous Approximation of the Score Function and Its Derivatives by Deep Neural Networks](https://mlsanity.com/paper/2512.23643/) (2025)
- [Denoising growth complexity: Data geometry and certified schedules for diffusion sampling](https://mlsanity.com/paper/2607.26285/) (2026)
- [ConvergeFlow: Language Flow with Provable Convergence to Token Embeddings](https://mlsanity.com/paper/2608.23551/) (2026)
- [Provably Reliable Classifier Guidance via Cross-Entropy Control](https://mlsanity.com/paper/2601.21200/) (2026)
- [On the Robustness of Distribution Support under Diffusion Guidance](https://mlsanity.com/paper/2605.07220/) (2026)
- [From Scores to Gibbs Correctors: Accelerating Uniform-Rate Discrete Diffusion Models](https://mlsanity.com/paper/2605.27352/) (2026)
- [A Quantitative Approximation Framework for Flow Distillation in Diffusion Models](https://mlsanity.com/paper/2606.03820/) (2026)
- [Diffusion Models Adapt to Low-Dimensional Structure Under Flexible Coefficient Choices](https://mlsanity.com/paper/2606.23627/) (2026)
- [Are First-Order Diffusion Samplers Really Slower? A Fast Forward-Value Approach](https://mlsanity.com/paper/2512.24927/) (2025)

## Abstract (arXiv)

In this paper, we explore provable acceleration of diffusion models without any additional retraining. Focusing on the task of approximating a target data distribution in ℝd to within varepsilon total-variation distance, we propose a principled, training-free sampling algorithm that requires only the order of  d1+2/K varepsilon-1/K  score function evaluations (up to log factor) in the presence of accurate scores, where K>0 is an arbitrary fixed integer. This result applies to a broad class of target data distributions, without the need for assumptions such as smoothness or log-concavity. Our theory is robust vis-a-vis inexact score estimation, degrading gracefully as the score estimation error increases -- without demanding higher-order smoothness on the score estimates as assumed in previous work. The proposed algorithm draws insight from high-order ODE solvers, leveraging high-order Lagrange interpolation and successive refinement to approximate the integral derived from the probability flow ODE. More broadly, our work develops a theoretical framework towards understanding the efficacy of high-order methods for accelerated sampling.