# Consistent Diffusion Models: Mitigating Sampling Drift by Learning to be Consistent

arXiv: https://arxiv.org/abs/2302.09057 · HTML: https://mlsanity.com/paper/2302.09057/

## Key facts

- Cited by 58 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 45 of its 57 references that are in the map.
- First posted on arXiv 17 Feb 2023.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 30 of 45)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [Video Diffusion Models](https://mlsanity.com/paper/2204.03458/) (2022)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [Imagen Video: High Definition Video Generation with Diffusion Models](https://mlsanity.com/paper/2210.02303/) (2022)
- [DiffWave: A Versatile Diffusion Model for Audio Synthesis](https://mlsanity.com/paper/2009.09761/) (2020)
- [Alias-Free Generative Adversarial Networks](https://mlsanity.com/paper/2106.12423/) (2021)
- [Variational Diffusion Models](https://mlsanity.com/paper/2107.00630/) (2021)
- [Improved Techniques for Training Score-Based Generative Models](https://mlsanity.com/paper/2006.09011/) (2020)
- [CogVideo: Large-scale Pretraining for Text-to-Video Generation via Transformers](https://mlsanity.com/paper/2205.15868/) (2022)
- [Sampling is as easy as learning the score: theory for diffusion models with minimal data assumptions](https://mlsanity.com/paper/2209.11215/) (2022)
- [Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioning](https://mlsanity.com/paper/2208.04202/) (2022)
- [PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models](https://mlsanity.com/paper/2003.03808/) (2020)
- [Understanding Diffusion Models: A Unified Perspective](https://mlsanity.com/paper/2208.11970/) (2022)
- [DiffDock: Diffusion Steps, Twists, and Turns for Molecular Docking](https://mlsanity.com/paper/2210.01776/) (2022)
- [Robust Compressed Sensing MRI with Deep Generative Priors](https://mlsanity.com/paper/2108.01368/) (2021)
- [Cold Diffusion: Inverting Arbitrary Image Transforms Without Noise](https://mlsanity.com/paper/2208.09392/) (2022)

## Built on by (most cited first, 30 of 58)

- [Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusion](https://mlsanity.com/paper/2310.02279/) (2023)
- [A Survey on Diffusion Models for Inverse Problems](https://mlsanity.com/paper/2410.00083/) (2024)
- [DDP: Diffusion Model for Dense Visual Prediction](https://mlsanity.com/paper/2303.17559/) (2023)
- [Ambient Diffusion: Learning Clean Distributions from Corrupted Data](https://mlsanity.com/paper/2305.19256/) (2023)
- [Flow map matching with stochastic interpolants: A mathematical framework for consistency models](https://mlsanity.com/paper/2406.07507/) (2024)
- [Fast ODE-based Sampling for Diffusion Models in Around 5 Steps](https://mlsanity.com/paper/2312.00094/) (2023)
- [Distillation of Discrete Diffusion through Dimensional Correlations](https://mlsanity.com/paper/2410.08709/) (2024)
- [CoMoSpeech: One-Step Speech and Singing Voice Synthesis via Consistency Model](https://mlsanity.com/paper/2305.06908/) (2023)
- [Consistent Diffusion Meets Tweedie: Training Exact Ambient Diffusion Models with Noisy Data](https://mlsanity.com/paper/2404.10177/) (2024)
- [DiffusionShield: A Watermark for Copyright Protection against Generative Diffusion Models](https://mlsanity.com/paper/2306.04642/) (2023)
- [The Journey, Not the Destination: How Data Guides Diffusion Models](https://mlsanity.com/paper/2312.06205/) (2023)
- [Solving Inverse Problems with Score-Based Generative Priors learned from Noisy Data](https://mlsanity.com/paper/2305.01166/) (2023)
- [Autoregressive Models in Vision: A Survey](https://mlsanity.com/paper/2411.05902/) (2024)
- [Ambient Diffusion Posterior Sampling: Solving Inverse Problems with Diffusion Models Trained on Corrupted Data](https://mlsanity.com/paper/2403.08728/) (2024)
- [Simple and Fast Distillation of Diffusion Models](https://mlsanity.com/paper/2409.19681/) (2024)
- [On Error Propagation of Diffusion Models](https://mlsanity.com/paper/2308.05021/) (2023)
- [VideoAgent: Self-Improving Video Generation](https://mlsanity.com/paper/2410.10076/) (2024)
- [Consistent Sampling and Simulation: Molecular Dynamics with Energy-Based Diffusion Models](https://mlsanity.com/paper/2506.17139/) (2025)
- [How much is a noisy image worth? Data Scaling Laws for Ambient Diffusion](https://mlsanity.com/paper/2411.02780/) (2024)
- [Ambient Diffusion Omni: Training Good Models with Bad Data](https://mlsanity.com/paper/2506.10038/) (2025)
- [Does Generation Require Memorization? Creative Diffusion Models using Ambient Diffusion](https://mlsanity.com/paper/2502.21278/) (2025)
- [Diffusion models for inverse problems](https://mlsanity.com/paper/2508.01975/) (2025)
- [FlexiDiT: Your Diffusion Transformer Can Easily Generate High-Quality Samples with Less Compute](https://mlsanity.com/paper/2502.20126/) (2025)
- [Diffusion for Natural Image Matting](https://mlsanity.com/paper/2312.05915/) (2023)
- [Structure-Guided Adversarial Training of Diffusion Models](https://mlsanity.com/paper/2402.17563/) (2024)
- [Stochastic Forward-Backward Deconvolution: Training Diffusion Models with Finite Noisy Datasets](https://mlsanity.com/paper/2502.05446/) (2025)
- [On the Equivalence of Consistency-Type Models: Consistency Models, Consistent Diffusion Models, and Fokker-Planck Regularization](https://mlsanity.com/paper/2306.00367/) (2023)
- [Synthetic Shifts to Initial Seed Vector Exposes the Brittle Nature of Latent-Based Diffusion Models](https://mlsanity.com/paper/2312.11473/) (2023)
- [Neural Approximate Mirror Maps for Constrained Diffusion Models](https://mlsanity.com/paper/2406.12816/) (2024)
- [Measurement Score-Based Diffusion Model](https://mlsanity.com/paper/2505.11853/) (2025)

## Abstract (arXiv)

Imperfect score-matching leads to a shift between the training and the sampling distribution of diffusion models. Due to the recursive nature of the generation process, errors in previous steps yield sampling iterates that drift away from the training distribution. Yet, the standard training objective via Denoising Score Matching (DSM) is only designed to optimize over non-drifted data. To train on drifted data, we propose to enforce a consistency property which states that predictions of the model on its own generated data are consistent across time. Theoretically, we show that if the score is learned perfectly on some non-drifted points (via DSM) and if the consistency property is enforced everywhere, then the score is learned accurately everywhere. Empirically we show that our novel training objective yields state-of-the-art results for conditional and unconditional generation in CIFAR-10 and baseline improvements in AFHQ and FFHQ. We open-source our code and models: https://github.com/giannisdaras/cdm