# Multistep Consistency Models

arXiv: https://arxiv.org/abs/2403.06807 · HTML: https://mlsanity.com/paper/2403.06807/

## Key facts

- Cited by 67 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 26 of its 28 references that are in the map.
- First posted on arXiv 11 Mar 2024.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 26 of 26)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023)
- [DiffWave: A Versatile Diffusion Model for Audio Synthesis](https://mlsanity.com/paper/2009.09761/) (2020)
- [Variational Diffusion Models](https://mlsanity.com/paper/2107.00630/) (2021)
- [One-step Diffusion with Distribution Matching Distillation](https://mlsanity.com/paper/2311.18828/) (2023)
- [On Distillation of Guided Diffusion Models](https://mlsanity.com/paper/2210.03142/) (2022)
- [Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusion](https://mlsanity.com/paper/2310.02279/) (2023)
- [Improved Techniques for Training Consistency Models](https://mlsanity.com/paper/2310.14189/) (2023)
- [Simple diffusion: End-to-end diffusion for high resolution images](https://mlsanity.com/paper/2301.11093/) (2023)
- [InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generation](https://mlsanity.com/paper/2309.06380/) (2023)
- [Diff-Instruct: A Universal Approach for Transferring Knowledge From Pre-trained Diffusion Models](https://mlsanity.com/paper/2305.18455/) (2023)
- [Gotta Go Fast When Generating Data with Score-Based Models](https://mlsanity.com/paper/2105.14080/) (2021)
- [Power Hungry Processing: Watts Driving the Cost of AI Deployment?](https://mlsanity.com/paper/2311.16863/) (2023)
- [Fast Sampling of Diffusion Models via Operator Learning](https://mlsanity.com/paper/2211.13449/) (2022)
- [UFOGen: You Forward Once Large Scale Text-to-Image Generation via Diffusion GANs](https://mlsanity.com/paper/2311.09257/) (2023)
- [TRACT: Denoising Diffusion Models with Transitive Closure Time-Distillation](https://mlsanity.com/paper/2303.04248/) (2023)

## Built on by (most cited first, 30 of 67)

- [Improved Distribution Matching Distillation for Fast Image Synthesis](https://mlsanity.com/paper/2405.14867/) (2024)
- [Mean Flows for One-step Generative Modeling](https://mlsanity.com/paper/2505.13447/) (2025)
- [From Slow Bidirectional to Fast Autoregressive Video Diffusion Models](https://mlsanity.com/paper/2412.07772/) (2024)
- [Simplifying, Stabilizing and Scaling Continuous-Time Consistency Models](https://mlsanity.com/paper/2410.11081/) (2024)
- [Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillation](https://mlsanity.com/paper/2403.12015/) (2024)
- [Consistency Models Made Easy](https://mlsanity.com/paper/2406.14548/) (2024)
- [Inductive Moment Matching](https://mlsanity.com/paper/2503.07565/) (2025)
- [SANA-Sprint: One-Step Diffusion with Continuous-Time Consistency Distillation](https://mlsanity.com/paper/2503.09641/) (2025)
- [Align Your Flow: Scaling Continuous-Time Flow Map Distillation](https://mlsanity.com/paper/2506.14603/) (2025)
- [Large Scale Diffusion Distillation via Score-Regularized Continuous-Time Consistency](https://mlsanity.com/paper/2510.08431/) (2025)
- [Phased Consistency Models](https://mlsanity.com/paper/2405.18407/) (2024)
- [Multistep Distillation of Diffusion Models via Moment Matching](https://mlsanity.com/paper/2406.04103/) (2024)
- [EM Distillation for One-step Diffusion Models](https://mlsanity.com/paper/2405.16852/) (2024)
- [Diffusion Models and Representation Learning: A Survey](https://mlsanity.com/paper/2407.00783/) (2024)
- [Adversarial Diffusion Compression for Real-World Image Super-Resolution](https://mlsanity.com/paper/2411.13383/) (2024)
- [Fast Solvers for Discrete Diffusion Models: Theory and Applications of High-Order Algorithms](https://mlsanity.com/paper/2502.00234/) (2025)
- [Accelerating Diffusion Models with Parallel Sampling: Inference at Sub-Linear Time Complexity](https://mlsanity.com/paper/2405.15986/) (2024)
- [Diff-Instruct++: Training One-step Text-to-image Generator Model to Align with Human Preferences](https://mlsanity.com/paper/2410.18881/) (2024)
- [Flow Generator Matching](https://mlsanity.com/paper/2410.19310/) (2024)
- [Efficient-vDiT: Efficient Video Diffusion Transformers With Attention Tile](https://mlsanity.com/paper/2502.06155/) (2025)
- [SplitMeanFlow: Interval Splitting Consistency in Few-Step Generative Modeling](https://mlsanity.com/paper/2507.16884/) (2025)
- [VideoAgent: Self-Improving Video Generation](https://mlsanity.com/paper/2410.10076/) (2024)
- [Truncated Consistency Models](https://mlsanity.com/paper/2410.14895/) (2024)
- [Decoupled MeanFlow: Turning Flow Models into Flow Maps for Accelerated Sampling](https://mlsanity.com/paper/2510.24474/) (2025)
- [CMT: Mid-Training for Efficient Learning of Consistency, Mean Flow, and Flow Map Models](https://mlsanity.com/paper/2509.24526/) (2025)
- [A Survey on Cache Methods in Diffusion Models: Toward Efficient Multi-Modal Generation](https://mlsanity.com/paper/2510.19755/) (2025)
- [VCT: Training Consistency Models with Variational Noise Coupling](https://mlsanity.com/paper/2502.18197/) (2025)
- [Invertible Consistency Distillation for Text-Guided Image Editing in Around 7 Steps](https://mlsanity.com/paper/2406.14539/) (2024)
- [Flow Matching Meets Biology and Life Science: A Survey](https://mlsanity.com/paper/2507.17731/) (2025)
- [Agentic World Modeling: Foundations, Capabilities, Laws, and Beyond](https://mlsanity.com/paper/2604.22748/) (2026)

## Abstract (arXiv)

Diffusion models are relatively easy to train but require many steps to generate samples. Consistency models are far more difficult to train, but generate samples in a single step. In this paper we propose Multistep Consistency Models: A unification between Consistency Models (Song et al., 2023) and TRACT (Berthelot et al., 2023) that can interpolate between a consistency model and a diffusion model: a trade-off between sampling speed and sampling quality. Specifically, a 1-step consistency model is a conventional consistency model whereas a ∞-step consistency model is a diffusion model. Multistep Consistency Models work really well in practice. By increasing the sample budget from a single step to 2-8 steps, we can train models more easily that generate higher quality samples, while retaining much of the sampling speed benefits. Notable results are 1.4 FID on Imagenet 64 in 8 step and 2.1 FID on Imagenet128 in 8 steps with consistency distillation, using simple losses without adversarial training. We also show that our method scales to a text-to-image diffusion model, generating samples that are close to the quality of the original model.