mlsanityData to 7 Oct 2026

Progressive Knowledge Distillation Of Stable Diffusion XL Using Layer Level Loss

arXiv 2401.02677 ↗First posted 5 Jan 2024cs.CV cs.AIcited by 51

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Stable Diffusion XL (SDXL) has become the best open source text-to-image model (T2I) for its versatility and top-notch image quality. Efficiently addressing the computational demands of SDXL models is crucial for wider reach and applicability. In this work, we introduce two scaled-down variants, Segmind Stable Diffusion (SSD-1B) and Segmind-Vega, with 1.3B and 0.74B parameter UNets, respectively, achieved through progressive removal using layer-level losses focusing on reducing the model size while preserving generative quality. We release these models weights at https://hf.co/Segmind. Our methodology involves the elimination of residual networks and transformer blocks from the U-Net structure of SDXL, resulting in significant reductions in parameters, and latency. Our compact models effectively emulate the original SDXL by capitalizing on transferred knowledge, achieving competitive results against larger multi-billion parameter SDXL. Our work underscores the efficacy of knowledge distillation coupled with layer-level losses in reducing model size while preserving the high-quality generative capabilities of SDXL, thus facilitating more accessible deployment in resource-constrained environments.

Key facts. Cited by 51 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 7 of its 14 references that are in the map. First posted on arXiv 5 Jan 2024.

Builds on

7 of 14 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference6 Oct
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec

Other references · 5

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  3. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  4. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,075
  5. 2023Wuerstchen: An Efficient Architecture for Large-Scale Text-to-Image Diffusion Modelscited by 50

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 21; 2025: 21; 2026: 9.

51 papers.

Sort:

By area:

  1. 2024FakeInversion: Learning to Detect Images from Unseen Text-to-Image Models by Inverting Stable Diffusioncited by 61
  2. 2024Faster Diffusion via Temporal Attention Decompositioncited by 54
  3. 2024LAPTOP-Diff: Layer Pruning and Normalized Distillation for Compressing Diffusion Modelscited by 52
  4. 2025Inference-time Scaling of Diffusion Models through Classical Searchcited by 48
  5. 2024FIRE: Robust Detection of Diffusion-Generated Images via Frequency-Guided Reconstruction Errorcited by 46
  6. 2025CO-SPY: Combining Semantic and Pixel Features to Detect Synthetic Images by AIcited by 42
  7. 2024A-Bench: Are LMMs Masters at Evaluating AI-generated Images?cited by 39
  8. 2024EvalMuse-40K: A Reliable and Fine-Grained Benchmark with Comprehensive Human Annotations for Text-to-Image Generation Model Evaluationcited by 37
  9. 2024AIGIQA-20K: A Large Database for AI-Generated Image Quality Assessmentcited by 33
  10. 2025The ML.ENERGY Benchmark: Toward Automated Inference Energy Measurement and Optimizationcited by 31
  11. 2024MISC: Ultra-low Bitrate Image Semantic Compression Driven by Large Multimodal Modelcited by 29
  12. 2024DiffCut: Catalyzing Zero-Shot Semantic Segmentation with Diffusion Features and Recursive Normalized Cutcited by 26
38 more
  1. 2024Large Multi-modality Model Assisted AI-Generated Image Quality Assessmentcited by 24
  2. 2025BlockDance: Reuse Structurally Similar Spatio-Temporal Features to Accelerate Diffusion Transformerscited by 24
  3. 2025Seg4Diff: Unveiling Open-Vocabulary Segmentation in Text-to-Image Diffusion Transformerscited by 24
  4. 2024SDXS: Real-Time One-Step Latent Diffusion Models with Image Conditionscited by 23
  5. 2024NTIRE 2024 Quality Assessment of AI-Generated Content Challengecited by 20
  6. 2024Reward Fine-Tuning Two-Step Diffusion Models via Learning Differentiable Latent-Space Surrogate Rewardcited by 16
  7. 2025Learning to Sample Effective and Diverse Prompts for Text-to-Image Generationcited by 16
  8. 2024Efficient Fine-Tuning and Concept Suppression for Pruned Diffusion Modelscited by 15
  9. 2024EdgeFusion: On-Device Text-to-Image Generationcited by 12
  10. 2025Image Quality Assessment: From Human to Machine Preferencecited by 12
  11. 2024ComfyGen: Prompt-Adaptive Workflows for Text-to-Image Generationcited by 11
  12. 2025LoRA-X: Bridging Foundation Models with Training-Free Cross-Model Adaptationcited by 10
  13. 2025Neodragon: Mobile Video Generation using Diffusion Transformercited by 10
  14. 2024CMC-Bench: Towards a New Paradigm of Visual Signal Compressioncited by 6
  15. 2025DRAGON: A Large-Scale Dataset of Realistic Images Generated by Diffusion Modelscited by 4
  16. 2024Unsupervised Segmentation by Diffusing, Walking and Cuttingcited by 3
  17. 2025DIP: Unsupervised Dense In-Context Post-training of Visual Representationscited by 3
  18. 2025EC-Diff: Fast and High-Quality Edge-Cloud Collaborative Inference for Diffusion Modelscited by 3
  19. 2026IncreFA: Breaking the Static Wall of Generative Model Attributioncited by 3
  20. 2025The Hidden Cost of an Image: Quantifying the Energy Consumption of AI Image Generationcited by 2
  21. 2025Memory-Efficient Personalization of Text-to-Image Diffusion Models via Selective Optimization Strategiescited by 2
  22. 2026A Difference-in-Difference Approach to Detecting AI-Generated Imagescited by 2
  23. 2024An Evaluation Framework for Product Images Background Inpainting based on Human Feedback and Product Consistencycited by 1
  24. 2025Temporally Consistent Unsupervised Segmentation for Mobile Robot Perceptioncited by 1
  25. 2025Match & Choose: Model Selection Framework for Fine-tuning Text-to-Image Diffusion Modelscited by 1
  26. 2026Boosting Visual Instruction Tuning with Self-Supervised Guidancecited by 1
  27. 2026Unlocking Diffusion Hierarchies: Adaptive Timestep Selection for Zero-Shot Segmentationcited by 1
  28. 2024DDIL: Diversity Enhancing Diffusion Distillation With Imitation Learningcited by 0
  29. 2024MAS-Attention: Memory-Aware Stream Processing for Attention Acceleration on Resource-Constrained Edge Devicescited by 0
  30. 2025Falcon: Fractional Alternating Cut with Overcoming Minima in Unsupervised Segmentationcited by 0
  31. 2025PromptIQ: Who Cares About Prompts? Let System Handle It -- A Component-Aware Framework for T2I Generationcited by 0
  32. 2025KDC-Diff: A Latent-Aware Diffusion Model with Knowledge Retention for Memory-Efficient Image Generationcited by 0
  33. 2025A Cram'er-von Mises Approach to Incentivizing Truthful Data Sharingcited by 0
  34. 2025REFINE-CONTROL: A Semi-supervised Distillation Method For Conditional Image Generationcited by 0
  35. 2026SPARK: Stochastic Propagation via Affinity-guided Random walK for training-free unsupervised segmentationcited by 0
  36. 2026LIFT and PLACE: A Simple, Stable, and Effective Knowledge Distillation Framework for Lightweight Diffusion Modelscited by 0
  37. 2026DynEval: Holistic Evaluations of T2I Generative Models in the Wildcited by 0
  38. 2026PixelUp: Zero-Shot Semantic Feature Upsampling for Fine-Grained Vision Taskscited by 0

50 most cited of 51 All 51 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis
  2. 2023A-SDM: Accelerating Stable Diffusion through Redundancy Removal and Performance Optimization
  3. 2023KOALA: Empirical Lessons Toward Memory-Efficient and Fast Diffusion Models for Text-to-Image Synthesis
  4. 2023BK-SDM: A Lightweight, Fast, and Cheap Version of Stable Diffusion
  5. 2024A-SDM: Accelerating Stable Diffusion through Model Assembly and Feature Inheritance Strategies
  6. 2025Few-Step Diffusion via Score identity Distillation
  7. 2025Delta Sampling: Data-Free Knowledge Transfer Across Diffusion Models
  8. 2024DKDM: Data-Free Knowledge Distillation for Diffusion Models with Any Architecture