mlsanityData to 7 Oct 2026

Multistep Consistency Models

arXiv 2403.06807 ↗First posted 11 Mar 2024cs.LG cs.CV stat.MLcited by 67

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion models are relatively easy to train but require many steps to generate samples. Consistency models are far more difficult to train, but generate samples in a single step. In this paper we propose Multistep Consistency Models: A unification between Consistency Models (Song et al., 2023) and TRACT (Berthelot et al., 2023) that can interpolate between a consistency model and a diffusion model: a trade-off between sampling speed and sampling quality. Specifically, a 1-step consistency model is a conventional consistency model whereas a ∞-step consistency model is a diffusion model. Multistep Consistency Models work really well in practice. By increasing the sample budget from a single step to 2-8 steps, we can train models more easily that generate higher quality samples, while retaining much of the sampling speed benefits. Notable results are 1.4 FID on Imagenet 64 in 8 step and 2.1 FID on Imagenet128 in 8 steps with consistency distillation, using simple losses without adversarial training. We also show that our method scales to a text-to-image diffusion model, generating samples that are close to the quality of the original model.

Key facts. Cited by 67 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 26 of its 28 references that are in the map. First posted on arXiv 11 Mar 2024.

Builds on

26 of 28 references.

Closest ancestors · 17

Same direction, newest first

  1. 2023One-step Diffusion with Distribution Matching Distillation30 Nov
  2. 2023Improved Techniques for Training Consistency Models22 Oct
  3. 2023Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusion1 Oct
  4. 2023Diff-Instruct: A Universal Approach for Transferring Knowledge From Pre-trained Diffusion Models29 May
  5. 2023TRACT: Denoising Diffusion Models with Transitive Closure Time-Distillation7 Mar
  6. 2023Consistency Models2 Mar
  7. 2023Simple diffusion: End-to-end diffusion for high resolution images26 Jan
  8. 2022Fast Sampling of Diffusion Models via Operator Learning24 Nov
  9. 2022On Distillation of Guided Diffusion Models6 Oct
  10. 2022Classifier-Free Diffusion Guidance26 Jul
  11. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  12. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
5 more
  1. 2021Variational Diffusion Models1 Jul
  2. 2021Gotta Go Fast When Generating Data with Score-Based Models28 May
  3. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  4. 2020Denoising Diffusion Implicit Models6 Oct
  5. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 9

  1. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  2. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  3. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  4. 2022Flow Matching for Generative Modelingcited by 5,426
  5. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
  6. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  7. 2023InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generationcited by 387
  8. 2023Power Hungry Processing: Watts Driving the Cost of AI Deployment?cited by 186
  9. 2023UFOGen: You Forward Once Large Scale Text-to-Image Generation via Diffusion GANscited by 165

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 24; 2025: 30; 2026: 13.

67 papers.

Sort:

By area:

  1. 2024Improved Distribution Matching Distillation for Fast Image Synthesiscited by 648
  2. 2025Mean Flows for One-step Generative Modelingcited by 524
  3. 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 401
  4. 2024Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillationcited by 276
  5. 2024Simplifying, Stabilizing and Scaling Continuous-Time Consistency Modelscited by 276
  6. 2024Consistency Models Made Easycited by 140
  7. 2025Inductive Moment Matchingcited by 107
  8. 2025SANA-Sprint: One-Step Diffusion with Continuous-Time Consistency Distillationcited by 85
  9. 2024Phased Consistency Modelscited by 83
  10. 2025Align Your Flow: Scaling Continuous-Time Flow Map Distillationcited by 81
  11. 2025Large Scale Diffusion Distillation via Score-Regularized Continuous-Time Consistencycited by 81
  12. 2024Multistep Distillation of Diffusion Models via Moment Matchingcited by 76
38 more
  1. 2024EM Distillation for One-step Diffusion Modelscited by 60
  2. 2024Diffusion Models and Representation Learning: A Surveycited by 53
  3. 2024Adversarial Diffusion Compression for Real-World Image Super-Resolutioncited by 49
  4. 2025Fast Solvers for Discrete Diffusion Models: Theory and Applications of High-Order Algorithmscited by 47
  5. 2024Accelerating Diffusion Models with Parallel Sampling: Inference at Sub-Linear Time Complexitycited by 42
  6. 2024Diff-Instruct++: Training One-step Text-to-image Generator Model to Align with Human Preferencescited by 30
  7. 2024Flow Generator Matchingcited by 26
  8. 2025Efficient-vDiT: Efficient Video Diffusion Transformers With Attention Tilecited by 26
  9. 2025SplitMeanFlow: Interval Splitting Consistency in Few-Step Generative Modelingcited by 26
  10. 2024VideoAgent: Self-Improving Video Generationcited by 24
  11. 2024Truncated Consistency Modelscited by 21
  12. 2025Decoupled MeanFlow: Turning Flow Models into Flow Maps for Accelerated Samplingcited by 19
  13. 2025CMT: Mid-Training for Efficient Learning of Consistency, Mean Flow, and Flow Map Modelscited by 18
  14. 2025VCT: Training Consistency Models with Variational Noise Couplingcited by 15
  15. 2024Invertible Consistency Distillation for Text-Guided Image Editing in Around 7 Stepscited by 14
  16. 2025Flow Matching Meets Biology and Life Science: A Surveycited by 14
  17. 2025A Survey on Cache Methods in Diffusion Models: Toward Efficient Multi-Modal Generationcited by 14
  18. 2024SSM Meets Video Diffusion Models: Efficient Long-Term Video Generation with Structured State Spacescited by 13
  19. 2024Improving Probabilistic Diffusion Models With Optimal Diagonal Covariance Matchingcited by 13
  20. 2025Improved Training Technique for Latent Consistency Modelscited by 13
  21. 2025Learning an Image Editing Model without Image Editing Pairscited by 13
  22. 2025SoFlow: Solution Flow Models for One-Step Generative Modelingcited by 12
  23. 2026Agentic World Modeling: Foundations, Capabilities, Laws, and Beyondcited by 10
  24. 2024Stable Consistency Tuning: Understanding and Improving Consistency Modelscited by 9
  25. 2024Multi-student Diffusion Distillation for Better One-step Generatorscited by 9
  26. 2024David and Goliath: Small One-step Model Beats Large Diffusion with Score Post-trainingcited by 8
  27. 2025DiffRatio: Training One-Step Diffusion Models Without Teacher Supervisioncited by 8
  28. 2025Learnable Sampler Distillation for Discrete Diffusion Modelscited by 8
  29. 2026Beyond Single Tokens: Distilling Discrete Diffusion Models via Discrete MMDcited by 8
  30. 2025Stream-DiffVSR: Low-Latency Streamable Video Super-Resolution via Auto-Regressive Diffusioncited by 7
  31. 2025Efficient Generative Model Training via Embedded Representation Warmupcited by 5
  32. 2025Compact Attention: Exploiting Structured Spatio-Temporal Sparsity for Fast Video Generationcited by 5
  33. 2026How to Guide Your Flow: Few-Step Alignment via Flow Map Reward Guidancecited by 5
  34. 2024TLCM: Training-efficient Latent Consistency Model for Image Generation with 2-8 Stepscited by 4
  35. 2026EditCtrl: Disentangled Local and Global Control for Real-Time Generative Video Editingcited by 4
  36. 2026Strong Stochastic Flow Mapscited by 4
  37. 2026Mean Flow Distillation: Robust and Stable Distillation for Flow Matching Modelscited by 3
  38. 2025Multi-Step Consistency Models: Fast Generation with Theoretical Guaranteescited by 2

50 most cited of 67 All 67 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Convergence Of Consistency Model With Multistep Sampling Under General Data Assumptions
  2. 2024Provable Statistical Rates for Consistency Diffusion Models
  3. 2024Towards a mathematical theory for consistency training in diffusion models
  4. 2023ACT-Diffusion: Efficient Adversarial Consistency Training for One-step Diffusion Models
  5. 2023Consistent Diffusion Models: Mitigating Sampling Drift by Learning to be Consistent
  6. 2023Convergence guarantee for consistency models
  7. 2024SCott: Accelerating Diffusion Models with Stochastic Consistency Distillation
  8. 2026Setting-Matched and Semantics-Scaled Benchmarking of One-Step Generative Models Against Multistep Diffusion and Flow Models