mlsanityData to 14 Oct 2026

Consistency Models Made Easy

arXiv 2406.14548 ↗First posted 20 Jun 2024cs.LG cs.CVcited by 144

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Consistency models (CMs) offer faster sampling than traditional diffusion models, but their training is resource-intensive. For example, as of 2024, training a state-of-the-art CM on CIFAR-10 takes one week on 8 GPUs. In this work, we propose an effective scheme for training CMs that largely improves the efficiency of building such models. Specifically, by expressing CM trajectories via a particular differential equation, we argue that diffusion models can be viewed as a special case of CMs. We can thus fine-tune a consistency model starting from a pretrained diffusion model and progressively approximate the full consistency condition to stronger degrees over the training process. Our resulting method, which we term Easy Consistency Tuning (ECT), achieves vastly reduced training times while improving upon the quality of previous methods: for example, ECT achieves a 2-step FID of 2.73 on CIFAR10 within 1 hour on a single A100 GPU, matching Consistency Distillation trained for hundreds of GPU hours. Owing to this computational efficiency, we investigate the scaling laws of CMs under ECT, showing that they obey the classic power law scaling, hinting at their ability to improve efficiency and performance at larger scales. Our code (https://github.com/locuslab/ect) is publicly available, making CMs more accessible to the broader community.

Key facts. Cited by 144 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 68 of its 84 references that are in the map. First posted on arXiv 20 Jun 2024.

Builds on

68 of 84 references.

Closest ancestors · 42

Same direction, newest first

  1. 2024Imagine Flash: Accelerating Emu Diffusion Models with Backward Distillation8 May
  2. 2024Score identity Distillation: Exponentially Fast Distillation of Pretrained Diffusion Models for One-Step Generation5 Apr
  3. 2024Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillation18 Mar
  4. 2024Multistep Consistency Models11 Mar
  5. 2024T-Stitch: Accelerating Sampling in Pre-Trained Diffusion Models with Trajectory Stitching21 Feb
  6. 2024SDXL-Lightning: Progressive Adversarial Diffusion Distillation21 Feb
  7. 2024Scalable High-Resolution Pixel-Space Image Synthesis with Hourglass Diffusion Transformers21 Jan
  8. 2023One-Step Diffusion Distillation via Deep Equilibrium Models12 Dec
  9. 2023Analyzing and Improving the Training Dynamics of Diffusion Models5 Dec
  10. 2023DiffiT: Diffusion Vision Transformers for Image Generation4 Dec
  11. 2023Fast ODE-based Sampling for Diffusion Models in Around 5 Steps30 Nov
  12. 2023One-step Diffusion with Distribution Matching Distillation30 Nov
30 more
  1. 2023Adversarial Diffusion Distillation28 Nov
  2. 2023Improved Techniques for Training Consistency Models22 Oct
  3. 2023DPM-Solver-v3: Improved Diffusion ODE Solver with Empirical Model Statistics20 Oct
  4. 2023Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference6 Oct
  5. 2023Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusion1 Oct
  6. 2023SA-Solver: Stochastic Adams Solver for Fast Sampling of Diffusion Models10 Sep
  7. 2023Convergence guarantee for consistency models22 Aug
  8. 2023BOOT: Data-free Distillation of Denoising Diffusion Models with Bootstrapping8 Jun
  9. 2023Diff-Instruct: A Universal Approach for Transferring Knowledge From Pre-trained Diffusion Models29 May
  10. 2023Parallel Sampling of Diffusion Models25 May
  11. 2023Efficient Diffusion Training via Min-SNR Weighting Strategy16 Mar
  12. 2023TRACT: Denoising Diffusion Models with Transitive Closure Time-Distillation7 Mar
  13. 2023Consistency Models2 Mar
  14. 2023Understanding Diffusion Objectives as the ELBO with Simple Data Augmentation1 Mar
  15. 2023UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Models9 Feb
  16. 2022Fast Sampling of Diffusion Models via Operator Learning24 Nov
  17. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models2 Nov
  18. 2022Deep Equilibrium Approaches to Diffusion Models23 Oct
  19. 2022On Distillation of Guided Diffusion Models6 Oct
  20. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps2 Jun
  21. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  22. 2022Fast Sampling of Diffusion Models with Exponential Integrator29 Apr
  23. 2022Pseudo Numerical Methods for Diffusion Models on Manifolds20 Feb
  24. 2021On Fast Sampling of Diffusion Probabilistic Models31 May
  25. 2021Cascaded Diffusion Models for High Fidelity Image Generation30 May
  26. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  27. 2021Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed7 Jan
  28. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  29. 2020Denoising Diffusion Implicit Models6 Oct
  30. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 26

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  4. 2022Flow Matching for Generative Modelingcited by 5,702
  5. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  6. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  7. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  8. 2020Exploring Simple Siamese Representation Learningcited by 2,866
  9. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  10. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,581
  11. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
  12. 2021Image Super-Resolution via Iterative Refinementcited by 1,511
14 more
  1. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,219
  2. 2020Training Generative Adversarial Networks with Limited Datacited by 1,186
  3. 2019On the Variance of the Adaptive Learning Rate and Beyondcited by 772
  4. 2022StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasetscited by 492
  5. 2024Lumiere: A Space-Time Diffusion Model for Video Generationcited by 455
  6. 2023Photorealistic Video Generation with Diffusion Modelscited by 306
  7. 2023Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion modelscited by 205
  8. 2023UFOGen: You Forward Once Large Scale Text-to-Image Generation via Diffusion GANscited by 166
  9. 2023SwiftBrush: One-Step Text-to-Image Diffusion Model with Variational Score Distillationcited by 110
  10. 2023PFGM++: Unlocking the Potential of Physics-Inspired Generative Modelscited by 73
  11. 2024DreamFlow: High-Quality Text-to-3D Generation by Approximating Probability Flowcited by 26
  12. 2024VP3D: Unleashing 2D Visual Prompt for Text-to-3D Generationcited by 25
  13. 2023HyperFields: Towards Zero-Shot Generation of NeRFs from Textcited by 13
  14. 2023CCM: Adding Conditional Controls to Text-to-Image Consistency Modelscited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 21; 2025: 61; 2026: 62.

144 papers.

Sort:

By area:

  1. 2025Mean Flows for One-step Generative Modelingcited by 561
  2. 2024One Step Diffusion via Shortcut Modelscited by 391
  3. 2024Simplifying, Stabilizing and Scaling Continuous-Time Consistency Modelscited by 283
  4. 2026Causal Forcing: Autoregressive Diffusion Distillation Done Right for High-Quality Real-Time Interactive Video Generationcited by 184
  5. 2025Inductive Moment Matchingcited by 113
  6. 2025Improved Mean Flows: On the Challenges of Fastforward Generative Modelscited by 110
  7. 2025How to build a consistency model: Learning flow maps via self-distillationcited by 103
  8. 2025Align Your Flow: Scaling Continuous-Time Flow Map Distillationcited by 87
  9. 2025Large Scale Diffusion Distillation via Score-Regularized Continuous-Time Consistencycited by 86
  10. 2024One-Step Diffusion Distillation through Score Implicit Matchingcited by 71
  11. 2024Rectified Diffusion: Straightness Is Not Your Need in Rectified Flowcited by 61
  12. 2026One-step Latent-free Image Generation with Pixel Mean Flowscited by 53
38 more
  1. 2025Is Noise Conditioning Necessary for Denoising Generative Models?cited by 44
  2. 2025One-step Diffusion Models with f-Divergence Distribution Matchingcited by 43
  3. 2025Adversarial Distribution Matching for Diffusion Distillation Towards Efficient Image and Video Synthesiscited by 41
  4. 2025AlphaFlow: Understanding and Improving MeanFlow Modelscited by 41
  5. 2024Simple ReFlow: Improved Techniques for Fast Flow Modelscited by 34
  6. 2024Music2Latent: Consistency Autoencoders for Latent Audio Compressioncited by 33
  7. 2024Diff-Instruct++: Training One-step Text-to-image Generator Model to Align with Human Preferencescited by 31
  8. 2026Transition Matching Distillation for Fast Video Generationcited by 30
  9. 2025VideoScene: Distilling Video Diffusion Model to Generate 3D Scenes in One Stepcited by 27
  10. 2024Flow Generator Matchingcited by 26
  11. 2026AnyFlow: Any-Step Video Diffusion Model with On-Policy Flow Map Distillationcited by 26
  12. 2025There is No VAE: End-to-End Pixel-Space Generative Modeling via Self-Supervised Pre-trainingcited by 24
  13. 2024Consistency Diffusion Bridge Modelscited by 23
  14. 2024Self-Guidance: Boosting Flow and Diffusion Generation on Their Owncited by 23
  15. 2024Truncated Consistency Modelscited by 22
  16. 2025DLM-One: Diffusion Language Models for One-Step Sequence Generationcited by 20
  17. 2025Decoupled MeanFlow: Turning Flow Models into Flow Maps for Accelerated Samplingcited by 20
  18. 2025Reward-Instruct: A Reward-Centric Approach to Fast Photo-Realistic Image Generationcited by 19
  19. 2026Flash-WAM: Modality-Aware Distillation for World Action Modelscited by 19
  20. 2025CMT: Mid-Training for Efficient Learning of Consistency, Mean Flow, and Flow Map Modelscited by 18
  21. 2025Visual Generation Without Guidancecited by 17
  22. 2025The Curse of Conditions: Analyzing and Improving Optimal Transport for Conditional Flow-Based Generationcited by 17
  23. 2025FACM: Flow-Anchored Consistency Modelscited by 16
  24. 2025Ultra-Fast Language Generation via Discrete Diffusion Divergence Instructcited by 16
  25. 2025VCT: Training Consistency Models with Variational Noise Couplingcited by 15
  26. 2025Fast Text-to-Audio Generation with Adversarial Post-Trainingcited by 14
  27. 2025Flow Matching Meets Biology and Life Science: A Surveycited by 14
  28. 2025Flow Map Distillation Without Datacited by 14
  29. 2025Improved Training Technique for Latent Consistency Modelscited by 13
  30. 2025Uni-Instruct: One-step Diffusion Model through Unified Diffusion Divergence Instructioncited by 13
  31. 2025Learning an Image Editing Model without Image Editing Pairscited by 13
  32. 2025FALCON: Few-step Accurate Likelihoods for Continuous Flowscited by 13
  33. 2025SoFlow: Solution Flow Models for One-Step Generative Modelingcited by 13
  34. 2026Representation Distribution Matching for One-Step Visual Generationcited by 11
  35. 2025S4S: Solving for a Diffusion Model Solvercited by 10
  36. 2025Learning to Integrate Diffusion ODEs by Averaging the Derivativescited by 10
  37. 2025LeMiCa: Lexicographic Minimax Path Caching for Efficient Diffusion-Based Video Generationcited by 10
  38. 2026MeanCache: From Instantaneous to Average Velocity for Accelerating Flow Matching Inferencecited by 10

50 most cited of 144 All 144 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Improving Consistency Models with Generator-Augmented Flows
  2. 2024Towards a mathematical theory for consistency training in diffusion models
  3. 2025Multi-Step Consistency Models: Fast Generation with Theoretical Guarantees
  4. 2025Convergence Of Consistency Model With Multistep Sampling Under General Data Assumptions
  5. 2025CDLM: Consistency Diffusion Language Models For Faster Sampling
  6. 2024SCott: Accelerating Diffusion Models with Stochastic Consistency Distillation
  7. 2024Provable Statistical Rates for Consistency Diffusion Models
  8. 2023Consistent Diffusion Models: Mitigating Sampling Drift by Learning to be Consistent