mlsanityData to 7 Oct 2026

Bespoke Non-Stationary Solvers for Fast Sampling of Diffusion and Flow Models

arXiv 2403.01329 ↗First posted 2 Mar 2024cs.LG cs.AI cs.CVcited by 9

Not on the map. Why?

Abstract (arXiv)

This paper introduces Bespoke Non-Stationary (BNS) Solvers, a solver distillation approach to improve sample efficiency of Diffusion and Flow models. BNS solvers are based on a family of non-stationary solvers that provably subsumes existing numerical ODE solvers and consequently demonstrate considerable improvement in sample approximation (PSNR) over these baselines. Compared to model distillation, BNS solvers benefit from a tiny parameter space (<200 parameters), fast optimization (two orders of magnitude faster), maintain diversity of samples, and in contrast to previous solver distillation approaches nearly close the gap from standard distillation methods such as Progressive Distillation in the low-medium NFE regime. For example, BNS solver achieves 45 PSNR / 1.76 FID using 16 NFE in class-conditional ImageNet-64. We experimented with BNS solvers for conditional image generation, text-to-image generation, and text-2-audio generation showing significant improvement in sample approximation (PSNR) in all.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 26 of its 37 references that are in the map. First posted on arXiv 2 Mar 2024.

Builds on

26 of 37 references.

References · 26

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  5. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  6. 2022Classifier-Free Diffusion Guidancecited by 5,717
  7. 2022Flow Matching for Generative Modelingcited by 5,426
  8. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  9. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  10. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  11. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,961
  12. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
14 more
  1. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
  2. 2021Variational Diffusion Modelscited by 1,145
  3. 2023One-step Diffusion with Distribution Matching Distillationcited by 978
  4. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Modelscited by 924
  5. 2022On Distillation of Guided Diffusion Modelscited by 721
  6. 2022Equivariant Diffusion for Molecule Generation in 3Dcited by 666
  7. 2022Fast Sampling of Diffusion Models with Exponential Integratorcited by 526
  8. 2017A Downsampled Variant of ImageNet as an Alternative to the CIFAR datasetscited by 374
  9. 2021Knowledge Distillation in Iterative Generative Models for Improved Sampling Speedcited by 344
  10. 2022Learning Fast Samplers for Diffusion Models by Differentiating Through Sample Qualitycited by 181
  11. 2023Guided Flows for Generative Modeling and Decision Makingcited by 99
  12. 2023EXPRESSO: A Benchmark and Analysis of Discrete Expressive Speech Resynthesiscited by 93
  13. 2023Training-free Linear Image Inverses via Flowscited by 70
  14. 2023Optimal Linear Subspace Search: Learning to Construct Fast and High-Quality Schedulers for Diffusion Modelscited by 12

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 6.

9 papers.

  1. 2024Video Diffusion Models: A Surveycited by 28
  2. 2025S4S: Solving for a Diffusion Model Solvercited by 10
  3. 2026Parallel Decoding Distillation for Fast Image and Video Generationcited by 7
  4. 2025Improving Diffusion-based Inverse Algorithms under Few-Step Constraint via Learnable Linear Extrapolationcited by 4
  5. 2026DyWeight: Dynamic Gradient Weighting for Few-Step Diffusion Samplingcited by 4
  6. 2026Few-Step Diffusion Sampling Through Instance-Aware Discretizationscited by 3
  7. 2026A universal vision transformer for fast calorimeter simulationscited by 2
  8. 2026Dual-Solver: A Generalized ODE Solver for Diffusion Models with Dual Predictioncited by 1
  9. 2026Learning to Solve Generative ODEs Beyond the Linear Spancited by 0