# Fast ODE-based Sampling for Diffusion Models in Around 5 Steps

arXiv: https://arxiv.org/abs/2312.00094 · HTML: https://mlsanity.com/paper/2312.00094/

## Key facts

- Cited by 84 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 36 of its 53 references that are in the map.
- First posted on arXiv 30 Nov 2023.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 36)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps](https://mlsanity.com/paper/2206.00927/) (2022)
- [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [Variational Diffusion Models](https://mlsanity.com/paper/2107.00630/) (2021)
- [DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models](https://mlsanity.com/paper/2211.01095/) (2022)
- [Pseudo Numerical Methods for Diffusion Models on Manifolds](https://mlsanity.com/paper/2202.09778/) (2022)
- [Fast Sampling of Diffusion Models with Exponential Integrator](https://mlsanity.com/paper/2204.13902/) (2022)
- [UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Models](https://mlsanity.com/paper/2302.04867/) (2023)
- [Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed](https://mlsanity.com/paper/2101.02388/) (2021)
- [Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.06503/) (2022)
- [Learning Fast Samplers for Diffusion Models by Differentiating Through Sample Quality](https://mlsanity.com/paper/2202.05830/) (2022)
- [GENIE: Higher-Order Denoising Diffusion Solvers](https://mlsanity.com/paper/2210.05475/) (2022)

## Built on by (most cited first, 30 of 84)

- [Consistency Models Made Easy](https://mlsanity.com/paper/2406.14548/) (2024)
- [On the Trajectory Regularity of ODE-based Diffusion Sampling](https://mlsanity.com/paper/2405.11326/) (2024)
- [Fast Solvers for Discrete Diffusion Models: Theory and Applications of High-Order Algorithms](https://mlsanity.com/paper/2502.00234/) (2025)
- [ROBIN: Robust and Invisible Watermarks for Diffusion Models with Adversarial Optimization](https://mlsanity.com/paper/2411.03862/) (2024)
- [Accelerating Diffusion Models with Parallel Sampling: Inference at Sub-Linear Time Complexity](https://mlsanity.com/paper/2405.15986/) (2024)
- [Learning to Discretize Denoising Diffusion ODEs](https://mlsanity.com/paper/2405.15506/) (2024)
- [Simple ReFlow: Improved Techniques for Fast Flow Models](https://mlsanity.com/paper/2410.07815/) (2024)
- [DCTdiff: Intriguing Properties of Image Generative Modeling in the DCT Space](https://mlsanity.com/paper/2412.15032/) (2024)
- [Simple and Fast Distillation of Diffusion Models](https://mlsanity.com/paper/2409.19681/) (2024)
- [SlimFlow: Training Smaller One-Step Diffusion Models with Rectified Flow](https://mlsanity.com/paper/2407.12718/) (2024)
- [Conditional Image Synthesis with Diffusion Models: A Survey](https://mlsanity.com/paper/2409.19365/) (2024)
- [Bidirectional Consistency Models](https://mlsanity.com/paper/2403.18035/) (2024)
- [Accelerating Image Generation with Sub-path Linear Approximation Model](https://mlsanity.com/paper/2404.13903/) (2024)
- [Latent Diffusion Model-Enabled Low-Latency Semantic Communication in the Presence of Semantic Ambiguities and Wireless Channel Noises](https://mlsanity.com/paper/2406.06644/) (2024)
- [Distilling Parallel Gradients for Fast ODE Solvers of Diffusion Models](https://mlsanity.com/paper/2507.14797/) (2025)
- [Physics-Informed Distillation of Diffusion Models for PDE-Constrained Generation](https://mlsanity.com/paper/2505.22391/) (2025)
- [On the Asymptotic Mean Square Error Optimality of Diffusion Models](https://mlsanity.com/paper/2403.02957/) (2024)
- [SHMT: Self-supervised Hierarchical Makeup Transfer via Latent Diffusion Models](https://mlsanity.com/paper/2412.11058/) (2024)
- [S4S: Solving for a Diffusion Model Solver](https://mlsanity.com/paper/2502.17423/) (2025)
- [Learning to Integrate Diffusion ODEs by Averaging the Derivatives](https://mlsanity.com/paper/2505.14502/) (2025)
- [PFDiff: Training-Free Acceleration of Diffusion Models Combining Past and Future Scores](https://mlsanity.com/paper/2408.08822/) (2024)
- [Learnable Sampler Distillation for Discrete Diffusion Models](https://mlsanity.com/paper/2509.19962/) (2025)
- [Two-Steps Diffusion Policy for Robotic Manipulation via Genetic Denoising](https://mlsanity.com/paper/2510.21991/) (2025)
- [Optimal Stepsize for Diffusion Sampling](https://mlsanity.com/paper/2503.21774/) (2025)
- [TCAQ-DM: Timestep-Channel Adaptive Quantization for Diffusion Models](https://mlsanity.com/paper/2412.16700/) (2024)
- [Optimizing for the Shortest Path in Denoising Diffusion Model](https://mlsanity.com/paper/2503.03265/) (2025)
- [Diffusion Model Quantization: A Review](https://mlsanity.com/paper/2505.05215/) (2025)
- [Revisiting Diffusion Models: From Generative Pre-training to One-Step Generation](https://mlsanity.com/paper/2506.09376/) (2025)
- [Frequency Regulation for Exposure Bias Mitigation in Diffusion Models](https://mlsanity.com/paper/2507.10072/) (2025)
- [Adaptive Stochastic Coefficients for Accelerating Diffusion Sampling](https://mlsanity.com/paper/2510.23285/) (2025)

## Abstract (arXiv)

Sampling from diffusion models can be treated as solving the corresponding ordinary differential equations (ODEs), with the aim of obtaining an accurate solution with as few number of function evaluations (NFE) as possible. Recently, various fast samplers utilizing higher-order ODE solvers have emerged and achieved better performance than the initial first-order one. However, these numerical methods inherently result in certain approximation errors, which significantly degrades sample quality with extremely small NFE (e.g., around 5). In contrast, based on the geometric observation that each sampling trajectory almost lies in a two-dimensional subspace embedded in the ambient space, we propose Approximate MEan-Direction Solver (AMED-Solver) that eliminates truncation errors by directly learning the mean direction for fast diffusion sampling. Besides, our method can be easily used as a plugin to further improve existing ODE-based samplers. Extensive experiments on image synthesis with the resolution ranging from 32 to 512 demonstrate the effectiveness of our method. With only 5 NFE, we achieve 6.61 FID on CIFAR-10, 10.74 FID on ImageNet 64×64, and 13.20 FID on LSUN Bedroom. Our code is available at https://github.com/zju-pi/diff-sampler.