# S4S: Solving for a Diffusion Model Solver arXiv: https://arxiv.org/abs/2502.17423 · HTML: https://mlsanity.com/paper/2502.17423/ ## Key facts - Cited by 10 papers in the map (data through 14 Oct 2026). - Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. - Builds on 35 of its 37 references that are in the map. - First posted on arXiv 24 Feb 2025. ## Placed in - Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/) ## Builds on (most cited first, 30 of 35) - [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020) - [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020) - [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020) - [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015) - [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022) - [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022) - [Diffusion Policy: Visuomotor Policy Learning via Action Diffusion](https://mlsanity.com/paper/2303.04137/) (2023) - [DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps](https://mlsanity.com/paper/2206.00927/) (2022) - [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023) - [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018) - [Variational Diffusion Models](https://mlsanity.com/paper/2107.00630/) (2021) - [DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models](https://mlsanity.com/paper/2211.01095/) (2022) - [On Distillation of Guided Diffusion Models](https://mlsanity.com/paper/2210.03142/) (2022) - [Pseudo Numerical Methods for Diffusion Models on Manifolds](https://mlsanity.com/paper/2202.09778/) (2022) - [Fast Sampling of Diffusion Models with Exponential Integrator](https://mlsanity.com/paper/2204.13902/) (2022) - [UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Models](https://mlsanity.com/paper/2302.04867/) (2023) - [Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions](https://mlsanity.com/paper/2209.11215/) (2022) - [Improved Techniques for Training Consistency Models](https://mlsanity.com/paper/2310.14189/) (2023) - [Diffusion Models Are Real-Time Game Engines](https://mlsanity.com/paper/2408.14837/) (2024) - [Reduce, Reuse, Recycle: Compositional Generation with Energy-Based Diffusion Models and MCMC](https://mlsanity.com/paper/2302.11552/) (2023) - [DPM-Solver-v3: Improved Diffusion ODE Solver with Empirical Model Statistics](https://mlsanity.com/paper/2310.13268/) (2023) - [Consistency Models Made Easy](https://mlsanity.com/paper/2406.14548/) (2024) - [Learning to Efficiently Sample from Diffusion Probabilistic Models](https://mlsanity.com/paper/2106.03802/) (2021) - [Align Your Steps: Optimizing Sampling Schedules in Diffusion Models](https://mlsanity.com/paper/2404.14507/) (2024) - [Fast ODE-based Sampling for Diffusion Models in Around 5 Steps](https://mlsanity.com/paper/2312.00094/) (2023) - [Accelerating Diffusion Sampling with Optimized Time Steps](https://mlsanity.com/paper/2402.17376/) (2024) - [On the Trajectory Regularity of ODE-based Diffusion Sampling](https://mlsanity.com/paper/2405.11326/) (2024) - [Learning to Discretize Denoising Diffusion ODEs](https://mlsanity.com/paper/2405.15506/) (2024) - [Bespoke Solvers for Generative Flow Models](https://mlsanity.com/paper/2310.19075/) (2023) - [Simple and Fast Distillation of Diffusion Models](https://mlsanity.com/paper/2409.19681/) (2024) ## Built on by (most cited first, 10 of 10) - [A Survey on Cache Methods in Diffusion Models: Toward Efficient Multi-Modal Generation](https://mlsanity.com/paper/2510.19755/) (2025) - [Learnable Sampler Distillation for Discrete Diffusion Models](https://mlsanity.com/paper/2509.19962/) (2025) - [Reward-Forcing: Autoregressive Video Generation with Reward Feedback](https://mlsanity.com/paper/2601.16933/) (2026) - [Geometric Regularity in Deterministic Sampling Dynamics of Diffusion-based Generative Models](https://mlsanity.com/paper/2506.10177/) (2025) - [DyWeight: Dynamic Gradient Weighting for Few-Step Diffusion Sampling](https://mlsanity.com/paper/2603.11607/) (2026) - [Hierarchical Schedule Optimization for Fast and Robust Diffusion Model Sampling](https://mlsanity.com/paper/2511.11688/) (2025) - [A Unified and Fast-Sampling Diffusion Bridge Framework via Stochastic Optimal Control](https://mlsanity.com/paper/2505.21528/) (2025) - [GAS: Improving Discretization of Diffusion ODEs via Generalized Adversarial Solver](https://mlsanity.com/paper/2510.17699/) (2025) - [Tracing the Oracle: Improving Diffusion Timestep Scheduling for 3D CT Reconstruction](https://mlsanity.com/paper/2606.06236/) (2026) - [Learning to Solve Generative ODEs Beyond the Linear Span](https://mlsanity.com/paper/2606.08672/) (2026) ## Abstract (arXiv) Diffusion models (DMs) create samples from a data distribution by starting from random noise and iteratively solving a reverse-time ordinary differential equation (ODE). Because each step in the iterative solution requires an expensive neural function evaluation (NFE), there has been significant interest in approximately solving these diffusion ODEs with only a few NFEs without modifying the underlying model. However, in the few NFE regime, we observe that tracking the true ODE evolution is fundamentally impossible using traditional ODE solvers. In this work, we propose a new method that learns a good solver for the DM, which we call Solving for the Solver (S4S). S4S directly optimizes a solver to obtain good generation quality by learning to match the output of a strong teacher solver. We evaluate S4S on six different pre-trained DMs, including pixel-space and latent-space DMs for both conditional and unconditional sampling. In all settings, S4S uniformly improves the sample quality relative to traditional ODE solvers. Moreover, our method is lightweight, data-free, and can be plugged in black-box on top of any discretization schedule or architecture to improve performance. Building on top of this, we also propose S4S-Alt, which optimizes both the solver and the discretization schedule. By exploiting the full design space of DM solvers, with 5 NFEs, we achieve an FID of 3.73 on CIFAR10 and 13.26 on MS-COCO, representing a 1.5× improvement over previous training-free ODE methods.