mlsanityData to 14 Oct 2026

KDC-Diff: A Latent-Aware Diffusion Model with Knowledge Retention for Memory-Efficient Image Generation

arXiv 2505.06995 ↗First posted 11 May 2025cs.CVcited by 0

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

The growing adoption of generative AI in real-world applications has exposed a critical bottleneck in the computational demands of diffusion-based text-to-image models. In this work, we propose KDC-Diff, a novel and scalable generative framework designed to significantly reduce computational overhead while maintaining high performance. At its core, KDC-Diff designs a structurally streamlined U-Net with a dual-layered knowledge distillation strategy to transfer semantic and structural representations from a larger teacher model. Moreover, a latent-space replay-based continual learning mechanism is incorporated to ensure stable generative performance across sequential tasks. Evaluated on benchmark datasets, our model demonstrates strong performance across FID, CLIP, KID, and LPIPS metrics while achieving substantial reductions in parameter count, inference time, and FLOPs. KDC-Diff offers a practical, lightweight, and generalizable solution for deploying diffusion models in low-resource environments, making it well-suited for the next generation of intelligent and resource-aware computing systems.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 32 of its 39 references that are in the map. First posted on arXiv 11 May 2025.

Builds on

32 of 39 references.

Closest ancestors · 8

Same direction, newest first

  1. 2024DKDM: Data-Free Knowledge Distillation for Diffusion Models with Any Architecture5 Sep
  2. 2024Adv-KD: Adversarial Knowledge Distillation for Faster Diffusion Sampling31 May
  3. 2024MixDQ: Memory-Efficient Few-Step Text-to-Image Diffusion Models with Metric-Decoupled Mixed Precision Quantization28 May
  4. 2024LiteVAE: Lightweight and Efficient Variational Autoencoders for Latent Diffusion Models23 May
  5. 2024Progressive Knowledge Distillation Of Stable Diffusion XL Using Layer Level Loss5 Jan
  6. 2023Diffusion with Forward Models: Solving Stochastic Inverse Problems Without Direct Supervision20 Jun
  7. 2023Q-Diffusion: Quantizing Diffusion Models8 Feb
  8. 2022Diffusion Models in Vision: A Survey10 Sep

Other references · 24

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,307
  3. 2021Perceiver: General Perception with Iterative Attentioncited by 1,104
  4. 2022Exploring CLIP for Assessing the Look and Feel of Imagescited by 991
  5. 2023Erasing Concepts from Diffusion Modelscited by 539
  6. 2023SVDiff: Compact Parameter Space for Diffusion Fine-Tuningcited by 338
  7. 2021Knowledge distillation: A good teacher is patient and consistentcited by 259
  8. 2023Continual Diffusion: Continual Customization of Text-to-Image Diffusion with C-LoRAcited by 141
  9. 2023MedSegDiff-V2: Diffusion based Medical Image Segmentation with Transformercited by 124
  10. 2023SwiftBrush: One-Step Text-to-Image Diffusion Model with Variational Score Distillationcited by 110
  11. 2021RECALL: Replay-based Continual Learning in Semantic Segmentationcited by 83
  12. 2023From Knowledge Distillation to Self-Knowledge Distillation: A Unified Approach with Normalized Loss and Customized Soft Labelscited by 69
12 more
  1. 2023MobileDiffusion: Instant Text-to-Image Generation on Mobile Devicescited by 42
  2. 2023The Power of Sound (TPoS): Audio Reactive Video Generation with Stable Diffusioncited by 41
  3. 2023Class-Incremental Learning using Diffusion Model for Distillation and Replaycited by 31
  4. 2021An Investigation of Replay-based Approaches for Continual Learningcited by 26
  5. 2024Diff-Tracker: Text-to-Image Diffusion Models are Unsupervised Trackerscited by 24
  6. 2024Diffusion-Driven Data Replay: A Novel Approach to Combat Forgetting in Federated Class Continual Learningcited by 23
  7. 2023The CLIP Model is Secretly an Image-to-Prompt Convertercited by 14
  8. 2023Edit Temporal-Consistent Videos with Image Diffusion Modelcited by 12
  9. 2022Comparison of Soft and Hard Target RNN-T Distillation for Large-scale ASRcited by 10
  10. 2021Compound Frechet Inception Distance for Quality Assessment of GAN Created Imagescited by 8
  11. 2020Topology Distance: A Topology-Based Approach For Evaluating Generative Adversarial Networkscited by 6
  12. 2022Cut Inner Layers: A Structured Pruning Strategy for Efficient U-Net GANscited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024Accelerating Diffusion Models with One-to-Many Knowledge Distillation
  2. 2023KOALA: Empirical Lessons Toward Memory-Efficient and Fast Diffusion Models for Text-to-Image Synthesis
  3. 2025Boosting Generative Image Modeling via Joint Image-Feature Synthesis
  4. 2023CoDi: Conditional Diffusion Distillation for Higher-Fidelity and Faster Image Generation
  5. 2026EruDiff: Refactoring Knowledge in Diffusion Models for Advanced Text-to-Image Synthesis
  6. 2025Data-free Knowledge Distillation with Diffusion Models
  7. 2023Continual Learning of Diffusion Models with Generative Distillation
  8. 2024On Improved Conditioning Mechanisms and Pre-training Strategies for Diffusion Models