# Differentiable Solver Search for Fast Diffusion Sampling

arXiv: https://arxiv.org/abs/2505.21114 · HTML: https://mlsanity.com/paper/2505.21114/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 36 of its 42 references that are in the map.
- First posted on arXiv 27 May 2025.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 36)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps](https://mlsanity.com/paper/2206.00927/) (2022)
- [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023)
- [MaskGIT: Masked Generative Image Transformer](https://mlsanity.com/paper/2202.04200/) (2022)
- [One-step Diffusion with Distribution Matching Distillation](https://mlsanity.com/paper/2311.18828/) (2023)
- [GenEval: An Object-Focused Framework for Evaluating Text-to-Image Alignment](https://mlsanity.com/paper/2310.11513/) (2023)
- [DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models](https://mlsanity.com/paper/2211.01095/) (2022)
- [SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers](https://mlsanity.com/paper/2401.08740/) (2024)
- [Fast Sampling of Diffusion Models with Exponential Integrator](https://mlsanity.com/paper/2204.13902/) (2022)
- [StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasets](https://mlsanity.com/paper/2202.00273/) (2022)
- [Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusion](https://mlsanity.com/paper/2310.02279/) (2023)
- [UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Models](https://mlsanity.com/paper/2302.04867/) (2023)
- [Symbolic Discovery of Optimization Algorithms](https://mlsanity.com/paper/2302.06675/) (2023)
- [SDXL-Lightning: Progressive Adversarial Diffusion Distillation](https://mlsanity.com/paper/2402.13929/) (2024)
- [Score identity Distillation: Exponentially Fast Distillation of Pretrained Diffusion Models for One-Step Generation](https://mlsanity.com/paper/2404.04057/) (2024)
- [DDT: Decoupled Diffusion Transformer](https://mlsanity.com/paper/2504.05741/) (2025)
- [Align Your Steps: Optimizing Sampling Schedules in Diffusion Models](https://mlsanity.com/paper/2404.14507/) (2024)
- [Distilling Diffusion Models into Conditional GANs](https://mlsanity.com/paper/2405.05967/) (2024)
- [Accelerating Diffusion Sampling with Optimized Time Steps](https://mlsanity.com/paper/2402.17376/) (2024)
- [On the Trajectory Regularity of ODE-based Diffusion Sampling](https://mlsanity.com/paper/2405.11326/) (2024)
- [Trajectory Consistency Distillation: Improved Latent Consistency Distillation by Semi-Linear Consistency Function with Trajectory Mapping](https://mlsanity.com/paper/2402.19159/) (2024)
- [Bespoke Solvers for Generative Flow Models](https://mlsanity.com/paper/2310.19075/) (2023)

## Built on by (most cited first, 6 of 6)

- [MotionRAG: Motion Retrieval-Augmented Image-to-Video Generation](https://mlsanity.com/paper/2509.26391/) (2025)
- [DyWeight: Dynamic Gradient Weighting for Few-Step Diffusion Sampling](https://mlsanity.com/paper/2603.11607/) (2026)
- [Error as Signal: Stiffness-Aware Diffusion Sampling via Embedded Runge-Kutta Guidance](https://mlsanity.com/paper/2603.03692/) (2026)
- [Dual-Solver: A Generalized ODE Solver for Diffusion Models with Dual Prediction](https://mlsanity.com/paper/2603.03973/) (2026)
- [Learning to Solve Generative ODEs Beyond the Linear Span](https://mlsanity.com/paper/2606.08672/) (2026)
- [TraceSampler: Making Sampling Errors Predictable and Correctable for Accelerated Generative Sampling](https://mlsanity.com/paper/2610.14332/) (2026)

## Abstract (arXiv)

Diffusion models have demonstrated remarkable generation quality but at the cost of numerous function evaluations. Recently, advanced ODE-based solvers have been developed to mitigate the substantial computational demands of reverse-diffusion solving under limited sampling steps. However, these solvers, heavily inspired by Adams-like multistep methods, rely solely on t-related Lagrange interpolation. We show that t-related Lagrange interpolation is suboptimal for diffusion model and reveal a compact search space comprised of time steps and solver coefficients. Building on our analysis, we propose a novel differentiable solver search algorithm to identify more optimal solver. Equipped with the searched solver, rectified-flow models, e.g., SiT-XL/2 and FlowDCN-XL/2, achieve FID scores of 2.40 and 2.35, respectively, on ImageNet256 with only 10 steps. Meanwhile, DDPM model, DiT-XL/2, reaches a FID score of 2.33 with only 10 steps. Notably, our searched solver outperforms traditional solvers by a significant margin. Moreover, our searched solver demonstrates generality across various model architectures, resolutions, and model sizes.