# Target-Driven Distillation: Consistency Distillation with Target Timestep Selection and Decoupled Guidance

arXiv: https://arxiv.org/abs/2409.01347 · HTML: https://mlsanity.com/paper/2409.01347/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models.
- Builds on 22 of its 33 references that are in the map.
- First posted on arXiv 2 Sep 2024.

## Placed in

- Diffusion models / [Diffusion foundations and sampling](https://mlsanity.com/diffusion/diffusion-foundations-and-sampling/)

## Builds on (most cited first, 22 of 22)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [One-step Diffusion with Distribution Matching Distillation](https://mlsanity.com/paper/2311.18828/) (2023)
- [Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference](https://mlsanity.com/paper/2310.04378/) (2023)
- [Fast Sampling of Diffusion Models with Exponential Integrator](https://mlsanity.com/paper/2204.13902/) (2022)
- [Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusion](https://mlsanity.com/paper/2310.02279/) (2023)
- [Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillation](https://mlsanity.com/paper/2403.12015/) (2024)
- [LayoutDiffusion: Controllable Diffusion Model for Layout-to-image Generation](https://mlsanity.com/paper/2303.17189/) (2023)
- [LCM-LoRA: A Universal Stable-Diffusion Acceleration Module](https://mlsanity.com/paper/2311.05556/) (2023)
- [Phased Consistency Models](https://mlsanity.com/paper/2405.18407/) (2024)
- [SphereDiffusion: Spherical Geometry-Aware Distortion Resilient Diffusion Model](https://mlsanity.com/paper/2403.10044/) (2024)

## Built on by (most cited first, 10 of 10)

- [FireRed-Image-Edit-1.0 Technical Report](https://mlsanity.com/paper/2602.13344/) (2026)
- [RestoreVAR: Visual Autoregressive Generation for All-in-One Image Restoration](https://mlsanity.com/paper/2505.18047/) (2025)
- [LeMiCa: Lexicographic Minimax Path Caching for Efficient Diffusion-Based Video Generation](https://mlsanity.com/paper/2511.00090/) (2025)
- [MeanCache: From Instantaneous to Average Velocity for Accelerating Flow Matching Inference](https://mlsanity.com/paper/2601.19961/) (2026)
- [Oscillation Inversion: Understand the structure of Large Flow Model through the Lens of Inversion Method](https://mlsanity.com/paper/2411.11135/) (2024)
- [Curvature-Adaptive Consistency Flow Matching: Autonomous Trajectory Optimization via Reinforcement Learning](https://mlsanity.com/paper/2606.22394/) (2026)
- [Shortcutting Pre-trained Flow Matching Diffusion Models is Almost Free Lunch](https://mlsanity.com/paper/2510.17858/) (2025)
- [Single Trajectory Distillation for Accelerating Image and Video Style Transfer](https://mlsanity.com/paper/2412.18945/) (2024)
- [OTCache: Optimal Transport for Geometry-Aware Caching in Diffusion Models](https://mlsanity.com/paper/2606.31026/) (2026)
- [Adaptive Sampling Scheduler](https://mlsanity.com/paper/2509.12569/) (2025)

## Abstract (arXiv)

Consistency distillation methods have demonstrated significant success in accelerating generative tasks of diffusion models. However, since previous consistency distillation methods use simple and straightforward strategies in selecting target timesteps, they usually struggle with blurs and detail losses in generated images. To address these limitations, we introduce Target-Driven Distillation (TDD), which (1) adopts a delicate selection strategy of target timesteps, increasing the training efficiency; (2) utilizes decoupled guidances during training, making TDD open to post-tuning on guidance scale during inference periods; (3) can be optionally equipped with non-equidistant sampling and x0 clipping, enabling a more flexible and accurate way for image sampling. Experiments verify that TDD achieves state-of-the-art performance in few-step generation, offering a better choice among consistency distillation models.