# SA-Solver: Stochastic Adams Solver for Fast Sampling of Diffusion Models

arXiv: https://arxiv.org/abs/2309.05019 · HTML: https://mlsanity.com/paper/2309.05019/

## Key facts

- Cited by 72 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 31 of its 44 references that are in the map.
- First posted on arXiv 10 Sep 2023.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 31)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [Video Diffusion Models](https://mlsanity.com/paper/2204.03458/) (2022)
- [DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps](https://mlsanity.com/paper/2206.00927/) (2022)
- [Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2304.08818/) (2023)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [Cascaded Diffusion Models for High Fidelity Image Generation](https://mlsanity.com/paper/2106.15282/) (2021)
- [Variational Diffusion Models](https://mlsanity.com/paper/2107.00630/) (2021)
- [DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models](https://mlsanity.com/paper/2211.01095/) (2022)
- [On Distillation of Guided Diffusion Models](https://mlsanity.com/paper/2210.03142/) (2022)
- [Fast Sampling of Diffusion Models with Exponential Integrator](https://mlsanity.com/paper/2204.13902/) (2022)
- [Tackling the Generative Learning Trilemma with Denoising Diffusion GANs](https://mlsanity.com/paper/2112.07804/) (2021)
- [UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Models](https://mlsanity.com/paper/2302.04867/) (2023)
- [Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed](https://mlsanity.com/paper/2101.02388/) (2021)
- [Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models](https://mlsanity.com/paper/2201.06503/) (2022)
- [Efficient Diffusion Training via Min-SNR Weighting Strategy](https://mlsanity.com/paper/2303.09556/) (2023)
- [Gotta Go Fast When Generating Data with Score-Based Models](https://mlsanity.com/paper/2105.14080/) (2021)
- [Diffusion-GAN: Training GANs with Diffusion](https://mlsanity.com/paper/2206.02262/) (2022)
- [Learning Fast Samplers for Diffusion Models by Differentiating Through Sample Quality](https://mlsanity.com/paper/2202.05830/) (2022)
- [BDDM: Bilateral Denoising Diffusion Models for Fast and High-Quality Speech Synthesis](https://mlsanity.com/paper/2203.13508/) (2022)
- [ERA-Solver: Error-Robust Adams Solver for Fast Sampling of Diffusion Probabilistic Models](https://mlsanity.com/paper/2301.12935/) (2023)

## Built on by (most cited first, 30 of 72)

- [PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis](https://mlsanity.com/paper/2310.00426/) (2023)
- [Score identity Distillation: Exponentially Fast Distillation of Pretrained Diffusion Models for One-Step Generation](https://mlsanity.com/paper/2404.04057/) (2024)
- [Consistency Models Made Easy](https://mlsanity.com/paper/2406.14548/) (2024)
- [Accelerating Convergence of Score-Based Diffusion Models, Provably](https://mlsanity.com/paper/2403.03852/) (2024)
- [One-Step Diffusion Distillation through Score Implicit Matching](https://mlsanity.com/paper/2410.16794/) (2024)
- [VideoSwap: Customized Video Subject Swapping with Interactive Semantic Point Correspondence](https://mlsanity.com/paper/2312.02087/) (2023)
- [Accelerating Diffusion Sampling with Optimized Time Steps](https://mlsanity.com/paper/2402.17376/) (2024)
- [Advantage Weighted Matching: Aligning RL with Pretraining in Diffusion Models](https://mlsanity.com/paper/2509.25050/) (2025)
- [The Blessing of Randomness: SDE Beats ODE in General Diffusion-based Image Editing](https://mlsanity.com/paper/2311.01410/) (2023)
- [Fast Solvers for Discrete Diffusion Models: Theory and Applications of High-Order Algorithms](https://mlsanity.com/paper/2502.00234/) (2025)
- [Stochastic Runge-Kutta Methods: Provable Acceleration of Diffusion Models](https://mlsanity.com/paper/2410.04760/) (2024)
- [Efficient Diffusion Models: A Survey](https://mlsanity.com/paper/2502.06805/) (2025)
- [DCTdiff: Intriguing Properties of Image Generative Modeling in the DCT Space](https://mlsanity.com/paper/2412.15032/) (2024)
- [Diff-Instruct++: Training One-step Text-to-image Generator Model to Align with Human Preferences](https://mlsanity.com/paper/2410.18881/) (2024)
- [Unifying Bayesian Flow Networks and Diffusion Models through Stochastic Differential Equations](https://mlsanity.com/paper/2404.15766/) (2024)
- [Provable Acceleration for Diffusion Models under Minimal Assumptions](https://mlsanity.com/paper/2410.23285/) (2024)
- [Towards a mathematical theory for consistency training in diffusion models](https://mlsanity.com/paper/2402.07802/) (2024)
- [Flow Generator Matching](https://mlsanity.com/paper/2410.19310/) (2024)
- [Efficient Diffusion Transformer with Step-wise Dynamic Attention Mediators](https://mlsanity.com/paper/2408.05710/) (2024)
- [Self-Guidance: Boosting Flow and Diffusion Generation on Their Own](https://mlsanity.com/paper/2412.05827/) (2024)
- [Reward-Instruct: A Reward-Centric Approach to Fast Photo-Realistic Image Generation](https://mlsanity.com/paper/2503.13070/) (2025)
- [HarmoniCa: Harmonizing Training and Inference for Better Feature Caching in Diffusion Transformer Acceleration](https://mlsanity.com/paper/2410.01723/) (2024)
- [Gaussian Mixture Flow Matching Models](https://mlsanity.com/paper/2504.05304/) (2025)
- [Consistency Trajectory Matching for One-Step Generative Super-Resolution](https://mlsanity.com/paper/2503.20349/) (2025)
- [Faster Diffusion Models via Higher-Order Approximation](https://mlsanity.com/paper/2506.24042/) (2025)
- [Ultra-Fast Language Generation via Discrete Diffusion Divergence Instruct](https://mlsanity.com/paper/2509.25035/) (2025)
- [Optimal Convergence Analysis of DDPM for General Distributions](https://mlsanity.com/paper/2510.27562/) (2025)
- [The Surprising Effectiveness of Skip-Tuning in Diffusion Sampling](https://mlsanity.com/paper/2402.15170/) (2024)
- [DC-Solver: Improving Predictor-Corrector Diffusion Sampler via Dynamic Compensation](https://mlsanity.com/paper/2409.03755/) (2024)
- [FlexiDiT: Your Diffusion Transformer Can Easily Generate High-Quality Samples with Less Compute](https://mlsanity.com/paper/2502.20126/) (2025)

## Abstract (arXiv)

Diffusion Probabilistic Models (DPMs) have achieved considerable success in generation tasks. As sampling from DPMs is equivalent to solving diffusion SDE or ODE which is time-consuming, numerous fast sampling methods built upon improved differential equation solvers are proposed. The majority of such techniques consider solving the diffusion ODE due to its superior efficiency. However, stochastic sampling could offer additional advantages in generating diverse and high-quality data. In this work, we engage in a comprehensive analysis of stochastic sampling from two aspects: variance-controlled diffusion SDE and linear multi-step SDE solver. Based on our analysis, we propose textitSA-Solver, which is an improved efficient stochastic Adams method for solving diffusion SDE to generate data with high quality. Our experiments show that textitSA-Solver achieves: 1) improved or comparable performance compared with the existing state-of-the-art (SOTA) sampling methods for few-step sampling; 2) SOTA FID on substantial benchmark datasets under a suitable number of function evaluations (NFEs). Code is available at https://github.com/scxue/SA-Solver.