# CoDi: Conditional Diffusion Distillation for Higher-Fidelity and Faster Image Generation

arXiv: https://arxiv.org/abs/2310.01407 · HTML: https://mlsanity.com/paper/2310.01407/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 49 of its 57 references that are in the map.
- First posted on arXiv 2 Oct 2023.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 30 of 49)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Parameter-Efficient Transfer Learning for NLP](https://mlsanity.com/paper/1902.00751/) (2019)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps](https://mlsanity.com/paper/2206.00927/) (2022)
- [SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations](https://mlsanity.com/paper/2108.01073/) (2021)
- [Consistency Models](https://mlsanity.com/paper/2303.01469/) (2023)
- [LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs](https://mlsanity.com/paper/2111.02114/) (2021)
- [Image Super-Resolution via Iterative Refinement](https://mlsanity.com/paper/2104.07636/) (2021)
- [Zero-1-to-3: Zero-shot One Image to 3D Object](https://mlsanity.com/paper/2303.11328/) (2023)
- [T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.08453/) (2023)
- [Palette: Image-to-Image Diffusion Models](https://mlsanity.com/paper/2111.05826/) (2021)
- [Variational Diffusion Models](https://mlsanity.com/paper/2107.00630/) (2021)
- [Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data](https://mlsanity.com/paper/2107.10833/) (2021)
- [DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models](https://mlsanity.com/paper/2211.01095/) (2022)
- [On Distillation of Guided Diffusion Models](https://mlsanity.com/paper/2210.03142/) (2022)
- [MagicBrush: A Manually Annotated Dataset for Instruction-Guided Image Editing](https://mlsanity.com/paper/2306.10012/) (2023)

## Built on by (most cited first, 26 of 26)

- [SnapGen-V: Generating a Five-Second Video within Five Seconds on a Mobile Device](https://mlsanity.com/paper/2412.10494/) (2024)
- [Fine-structure Preserved Real-world Image Super-resolution via Transfer VAE Training](https://mlsanity.com/paper/2507.20291/) (2025)
- [Bigger is not Always Better: Scaling Properties of Latent Diffusion Models](https://mlsanity.com/paper/2404.01367/) (2024)
- [The Power of Context: How Multimodality Improves Image Super-Resolution](https://mlsanity.com/paper/2503.14503/) (2025)
- [Improving the Stability and Efficiency of Diffusion Models for Content Consistent Super-Resolution](https://mlsanity.com/paper/2401.00877/) (2023)
- [RayFlow: Instance-Aware Diffusion Acceleration via Adaptive Flow Trajectories](https://mlsanity.com/paper/2503.07699/) (2025)
- [Taming Diffusion Transformer for Efficient Mobile Video Generation in Seconds](https://mlsanity.com/paper/2507.13343/) (2025)
- [AdaDiffSR: Adaptive Region-aware Dynamic Acceleration Diffusion Model for Real-World Image Super-Resolution](https://mlsanity.com/paper/2410.17752/) (2024)
- [Learning Straight Flows: Variational Flow Matching for Efficient Generation](https://mlsanity.com/paper/2511.17583/) (2025)
- [Edge-SD-SR: Low Latency and Parameter Efficient On-device Super-Resolution with Stable Diffusion via Bidirectional Conditioning](https://mlsanity.com/paper/2412.06978/) (2024)
- [Text-Aware Image Restoration with Diffusion Models](https://mlsanity.com/paper/2506.09993/) (2025)
- [UniRes: Universal Image Restoration for Complex Degradations](https://mlsanity.com/paper/2506.05599/) (2025)
- [HFMF: Hierarchical Fusion Meets Multi-Stream Models for Deepfake Detection](https://mlsanity.com/paper/2501.05631/) (2025)
- [GuideSR: Rethinking Guidance for One-Step High-Fidelity Diffusion-Based Super-Resolution](https://mlsanity.com/paper/2505.00687/) (2025)
- [Guidance Free Image Editing via Explicit Conditioning](https://mlsanity.com/paper/2503.17593/) (2025)
- [Model-Agnostic Gender Bias Control for Text-to-Image Generation via Sparse Autoencoder](https://mlsanity.com/paper/2507.20973/) (2025)
- [Unified Diffusion Transformer for High-fidelity Text-Aware Image Restoration](https://mlsanity.com/paper/2512.08922/) (2025)
- [Uni-DAD: Unified Distillation and Adaptation of Diffusion Models for Few-step Few-shot Image Generation](https://mlsanity.com/paper/2511.18281/) (2025)
- [3D-Consistent Multi-View Editing by Correspondence Guidance](https://mlsanity.com/paper/2511.22228/) (2025)
- [Multi-scale Image Super Resolution with a Single Auto-Regressive Model](https://mlsanity.com/paper/2506.04990/) (2025)
- [Generative Manifold Distillation: Aligning Restoration Trajectories with Natural Image Prior](https://mlsanity.com/paper/2512.11121/) (2025)
- [FlowBlending: Stage-Aware Multi-Model Sampling for Fast and High-Fidelity Video Generation](https://mlsanity.com/paper/2512.24724/) (2025)
- [Beyond Fixed Inference: Quantitative Flow Matching for Adaptive Image Denoising](https://mlsanity.com/paper/2604.02392/) (2026)
- [DGSSM: Diffusion guided state-space models for multimodal salient object detection](https://mlsanity.com/paper/2604.17585/) (2026)
- [Hierarchical Image Tokenization for Multi-Scale Image Super Resolution](https://mlsanity.com/paper/2605.14891/) (2026)
- [Rethinking Conditional Generation for Underwater Salient Object Detection](https://mlsanity.com/paper/2607.01825/) (2026)

## Abstract (arXiv)

Large generative diffusion models have revolutionized text-to-image generation and offer immense potential for conditional generation tasks such as image enhancement, restoration, editing, and compositing. However, their widespread adoption is hindered by the high computational cost, which limits their real-time application. To address this challenge, we introduce a novel method dubbed CoDi, that adapts a pre-trained latent diffusion model to accept additional image conditioning inputs while significantly reducing the sampling steps required to achieve high-quality results. Our method can leverage architectures such as ControlNet to incorporate conditioning inputs without compromising the model's prior knowledge gained during large scale pre-training. Additionally, a conditional consistency loss enforces consistent predictions across diffusion steps, effectively compelling the model to generate high-quality images with conditions in a few steps. Our conditional-task learning and distillation approach outperforms previous distillation methods, achieving a new state-of-the-art in producing high-quality images with very few steps (e.g., 1-4) across multiple tasks, including super-resolution, text-guided image editing, and depth-to-image generation.