# Bespoke Solvers for Generative Flow Models

arXiv: https://arxiv.org/abs/2310.19075 · HTML: https://mlsanity.com/paper/2310.19075/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 39 references that are in the map.
- First posted on arXiv 29 Oct 2023.

## Builds on (most cited first, 29 of 29)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Neural Ordinary Differential Equations](https://mlsanity.com/paper/1806.07366/) (2018)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps](https://mlsanity.com/paper/2206.00927/) (2022)
- [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023)
- [DiffWave: A Versatile Diffusion Model for Audio Synthesis](https://mlsanity.com/paper/2009.09761/) (2020)
- [Variational Diffusion Models](https://mlsanity.com/paper/2107.00630/) (2021)
- [StarGAN v2: Diverse Image Synthesis for Multiple Domains](https://mlsanity.com/paper/1912.01865/) (2019)
- [DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models](https://mlsanity.com/paper/2211.01095/) (2022)
- [On Distillation of Guided Diffusion Models](https://mlsanity.com/paper/2210.03142/) (2022)
- [Fast Sampling of Diffusion Models with Exponential Integrator](https://mlsanity.com/paper/2204.13902/) (2022)
- [Voicebox: Text-Guided Multilingual Universal Speech Generation at Scale](https://mlsanity.com/paper/2306.15687/) (2023)
- [A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasets](https://mlsanity.com/paper/1707.08819/) (2017)
- [Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed](https://mlsanity.com/paper/2101.02388/) (2021)
- [Learning Fast Samplers for Diffusion Models by Differentiating Through Sample Quality](https://mlsanity.com/paper/2202.05830/) (2022)
- [Fast Sampling of Diffusion Models via Operator Learning](https://mlsanity.com/paper/2211.13449/) (2022)
- [GENIE: Higher-Order Denoising Diffusion Solvers](https://mlsanity.com/paper/2210.05475/) (2022)
- [Diffusion Probabilistic Model Made Slim](https://mlsanity.com/paper/2211.17106/) (2022)
- [Bilateral Denoising Diffusion Models](https://mlsanity.com/paper/2108.11514/) (2021)
- [Improved Order Analysis and Design of Exponential Integrator for Diffusion Models Sampling](https://mlsanity.com/paper/2308.02157/) (2023)
- [Optimal Linear Subspace Search: Learning to Construct Fast and High-Quality Schedulers for Diffusion Models](https://mlsanity.com/paper/2305.14677/) (2023)

## Built on by (most cited first, 30 of 37)

- [Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillation](https://mlsanity.com/paper/2403.12015/) (2024)
- [Cache Me if You Can: Accelerating Diffusion Models through Block Caching](https://mlsanity.com/paper/2312.03209/) (2023)
- [Audiobox: Unified Audio Generation with Natural Language Prompts](https://mlsanity.com/paper/2312.15821/) (2023)
- [Improving the Training of Rectified Flows](https://mlsanity.com/paper/2405.20320/) (2024)
- [Neural Flow Diffusion Models: Learnable Forward Process for Improved Diffusion Modelling](https://mlsanity.com/paper/2404.12940/) (2024)
- [Inference-Time Scaling for Flow Models via Stochastic Generation and Rollover Budget Forcing](https://mlsanity.com/paper/2503.19385/) (2025)
- [Imagine Flash: Accelerating Emu Diffusion Models with Backward Distillation](https://mlsanity.com/paper/2405.05224/) (2024)
- [Adaptive Guidance: Training-free Acceleration of Conditional Diffusion Models](https://mlsanity.com/paper/2312.12487/) (2023)
- [Diff2Flow: Training Flow Matching Models via Diffusion Model Alignment](https://mlsanity.com/paper/2506.02221/) (2025)
- [From discrete-time policies to continuous-time diffusion samplers: Asymptotic equivalences and faster training](https://mlsanity.com/paper/2501.06148/) (2025)
- [Fast Samplers for Inverse Problems in Iterative Refinement Models](https://mlsanity.com/paper/2405.17673/) (2024)
- [Lipschitz-Guided Design of Interpolation Schedules in Generative Models](https://mlsanity.com/paper/2509.01629/) (2025)
- [GLASS Flows: Transition Sampling for Alignment of Flow and Diffusion Models](https://mlsanity.com/paper/2509.25170/) (2025)
- [MusicFlow: Cascaded Flow Matching for Text Guided Music Generation](https://mlsanity.com/paper/2410.20478/) (2024)
- [S4S: Solving for a Diffusion Model Solver](https://mlsanity.com/paper/2502.17423/) (2025)
- [Improving Rectified Flow with Boundary Conditions](https://mlsanity.com/paper/2506.15864/) (2025)
- [Hamiltonian Score Matching and Generative Flows](https://mlsanity.com/paper/2410.20470/) (2024)
- [B'ezierFlow: Learning B'ezier Stochastic Interpolant Schedulers for Few-Step Generation](https://mlsanity.com/paper/2512.13255/) (2025)
- [Faster Inference of Flow-Based Generative Models via Improved Data-Noise Coupling](https://mlsanity.com/paper/2603.15279/) (2026)
- [Taming Flow Matching with Unbalanced Optimal Transport into Fast Pansharpening](https://mlsanity.com/paper/2503.14975/) (2025)
- [Differentiable Solver Search for Fast Diffusion Sampling](https://mlsanity.com/paper/2505.21114/) (2025)
- [Scale-Adaptive Generative Flows for Multiscale Scientific Data](https://mlsanity.com/paper/2509.02971/) (2025)
- [FlowCast: Trajectory Forecasting for Scalable Zero-Cost Speculative Flow Matching](https://mlsanity.com/paper/2602.01329/) (2026)
- [FastFlow: Accelerating The Generative Flow Matching Models with Bandit Inference](https://mlsanity.com/paper/2602.11105/) (2026)
- [Few-Step Diffusion Sampling Through Instance-Aware Discretizations](https://mlsanity.com/paper/2603.17671/) (2026)
- [Animated Stickers: Bringing Stickers to Life with Video Diffusion](https://mlsanity.com/paper/2402.06088/) (2024)
- [A unified perspective on fine-tuning and sampling with diffusion and flow models](https://mlsanity.com/paper/2605.00229/) (2026)
- [Boosting Diffusion Model for Spectrogram Up-sampling in Text-to-speech: An Empirical Study](https://mlsanity.com/paper/2406.04633/) (2024)
- [Error as Signal: Stiffness-Aware Diffusion Sampling via Embedded Runge-Kutta Guidance](https://mlsanity.com/paper/2603.03692/) (2026)
- [Dual-Solver: A Generalized ODE Solver for Diffusion Models with Dual Prediction](https://mlsanity.com/paper/2603.03973/) (2026)

## Abstract (arXiv)

Diffusion or flow-based models are powerful generative paradigms that are notoriously hard to sample as samples are defined as solutions to high-dimensional Ordinary or Stochastic Differential Equations (ODEs/SDEs) which require a large Number of Function Evaluations (NFE) to approximate well. Existing methods to alleviate the costly sampling process include model distillation and designing dedicated ODE solvers. However, distillation is costly to train and sometimes can deteriorate quality, while dedicated solvers still require relatively large NFE to produce high quality samples. In this paper we introduce "Bespoke solvers", a novel framework for constructing custom ODE solvers tailored to the ODE of a given pre-trained flow model. Our approach optimizes an order consistent and parameter-efficient solver (e.g., with 80 learnable parameters), is trained for roughly 1% of the GPU time required for training the pre-trained model, and significantly improves approximation and generation quality compared to dedicated solvers. For example, a Bespoke solver for a CIFAR10 model produces samples with Fr'echet Inception Distance (FID) of 2.73 with 10 NFE, and gets to 1% of the Ground Truth (GT) FID (2.59) for this model with only 20 NFE. On the more challenging ImageNet-64×64, Bespoke samples at 2.2 FID with 10 NFE, and gets within 2% of GT FID (1.71) with 20 NFE.