mlsanityData to 7 Oct 2026

Efficient Diffusion Models: A Survey

arXiv 2502.06805 ↗First posted 3 Feb 2025cs.LG cs.GRcited by 35

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion models have emerged as powerful generative models capable of producing high-quality contents such as images, videos, and audio, demonstrating their potential to revolutionize digital content creation. However, these capabilities come at the cost of their significant computational resources and lengthy generation time, underscoring the critical need to develop efficient techniques for practical deployment. In this survey, we provide a systematic and comprehensive review of research on efficient diffusion models. We organize the literature in a taxonomy consisting of three main categories, covering distinct yet interconnected efficient diffusion model topics from algorithm-level, system-level, and framework perspective, respectively. We have also created a GitHub repository where we organize the papers featured in this survey at https://github.com/AIoT-MLSys-Lab/Efficient-Diffusion-Model-Survey. We hope our survey can serve as a valuable resource to help researchers and practitioners gain a systematic understanding of efficient diffusion model research and inspire them to contribute to this important and exciting field.

Key facts. Cited by 35 papers in the map (data through 7 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 167 of its 197 references that are in the map. First posted on arXiv 3 Feb 2025.

Builds on

167 of 197 references.

Closest ancestors · 75

Same direction, newest first

  1. 2025Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Models2 Jan
  2. 2024Unveiling Redundancy in Diffusion Transformers (DiTs): A Systematic Study18 Nov
  3. 2024xDiT: an Inference Engine for Diffusion Transformers (DiTs) with Massive Parallelism4 Nov
  4. 2024Efficient Diffusion Models: A Comprehensive Survey from Principles to Practices15 Oct
  5. 2024Simplifying, Stabilizing and Scaling Continuous-Time Consistency Models14 Oct
  6. 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think9 Oct
  7. 2024A Simple Early Exiting Framework for Accelerated Sampling in Diffusion Models12 Aug
  8. 2024SlimFlow: Training Smaller One-Step Diffusion Models with Rectified Flow17 Jul
  9. 2024Improved Noise Schedule for Diffusion Training3 Jul
  10. 2024SwiftDiffusion: Efficient Diffusion Model Serving with Add-on Modules2 Jul
  11. 2024FORA: Fast-Forward Caching in Diffusion Transformer Acceleration1 Jul
  12. 2024Immiscible Diffusion: Accelerating Diffusion Training with Noise Assignment18 Jun
38 more
  1. 2024Learning-to-Cache: Accelerating Diffusion Transformer via Layer Caching3 Jun
  2. 2024PipeFusion: Patch-level Pipeline Parallelism for Diffusion Transformers Inference23 May
  3. 2024Simple Drop-in LoRA Conditioning on Attention Layers Will Improve Your Diffusion Model7 May
  4. 2024Unifying Bayesian Flow Networks and Diffusion Models through Stochastic Differential Equations24 Apr
  5. 2024Align Your Steps: Optimizing Sampling Schedules in Diffusion Models22 Apr
  6. 2024LD-Pruner: Efficient Pruning of Latent Diffusion Models using Task-Agnostic Insights18 Apr
  7. 2024LAPTOP-Diff: Layer Pruning and Normalized Distillation for Compressing Diffusion Models17 Apr
  8. 2024An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimization11 Apr
  9. 2024DistriFusion: Distributed Parallel Inference for High-Resolution Diffusion Models29 Feb
  10. 2024Accelerating Parallel Sampling of Diffusion Models15 Feb
  11. 2023Non-Denoising Forward-Time Diffusions22 Dec
  12. 2023Cache Me if You Can: Accelerating Diffusion Models through Block Caching6 Dec
  13. 2023DeepCache: Accelerating Diffusion Models for Free1 Dec
  14. 2023One-step Diffusion with Distribution Matching Distillation30 Nov
  15. 2023AdaDiff: Adaptive Step Selection for Fast Diffusion Models24 Nov
  16. 2023LCM-LoRA: A Universal Stable-Diffusion Acceleration Module9 Nov
  17. 2023Gaussian Mixture Solvers for Diffusion Models2 Nov
  18. 2023Improved Techniques for Training Consistency Models22 Oct
  19. 2023Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference6 Oct
  20. 2023EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Diffusion Models5 Oct
  21. 2023SA-Solver: Stochastic Adams Solver for Fast Sampling of Diffusion Models10 Sep
  22. 2023Diffusion Models with Deterministic Normalizing Flow Priors3 Sep
  23. 2023Restart Sampling for Improving Generative Processes26 Jun
  24. 2023Temporal Dynamic Quantization for Diffusion Models4 Jun
  25. 2023Diff-Instruct: A Universal Approach for Transferring Knowledge From Pre-trained Diffusion Models29 May
  26. 2023Parallel Sampling of Diffusion Models25 May
  27. 2023Structural Pruning for Diffusion Models18 May
  28. 2023PTQD: Accurate Post-Training Quantization for Diffusion Models18 May
  29. 2023Improved Techniques for Maximum Likelihood Estimation for Diffusion ODEs6 May
  30. 2023Iterative α-(de)Blending: a Minimalist Deterministic Diffusion Model5 May
  31. 2023Speed Is All You Need: On-Device Acceleration of Large Diffusion Models via GPU-Aware Optimizations21 Apr
  32. 2023A Comprehensive Survey on Knowledge Distillation of Diffusion Models9 Apr
  33. 2023Consistency Models2 Mar
  34. 2023Q-Diffusion: Quantizing Diffusion Models8 Feb
  35. 2023ReDi: Efficient Learning-Free Diffusion Inference via Trajectory Retrieval5 Feb
  36. 2022Scalable Diffusion Models with Transformers19 Dec
  37. 2022Post-training Quantization on Diffusion Models28 Nov
  38. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models2 Nov

50 newest of 75 All 75 (JSON)

Other references · 92

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  6. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  7. 2022Flow Matching for Generative Modelingcited by 5,426
  8. 2017Neural Discrete Representation Learningcited by 4,650
  9. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,067
  10. 2018Neural Ordinary Differential Equationscited by 3,733
  11. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  12. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
38 more
  1. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,355
  2. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
  3. 2022Video Diffusion Modelscited by 2,280
  4. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  5. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,439
  6. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,339
  7. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,339
  8. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,207
  9. 2022Diffusion-LM Improves Controllable Text Generationcited by 1,006
  10. 2023Stochastic Interpolants: A Unifying Framework for Flows and Diffusionscited by 897
  11. 2022DiffEdit: Diffusion-based semantic image editing with mask guidancecited by 643
  12. 2020WaveGrad: Estimating Gradients for Waveform Generationcited by 620
  13. 2019Learned Step Size Quantizationcited by 604
  14. 2023AudioLDM: Text-to-Audio Generation with Latent Diffusion Modelscited by 592
  15. 2022Executing your Commands via Motion Diffusion in Latent Spacecited by 567
  16. 2022DiGress: Discrete Denoising diffusion for graph generationcited by 503
  17. 2021BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstructioncited by 464
  18. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Modelscited by 463
  19. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Modelscited by 439
  20. 2023InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generationcited by 387
  21. 2022Latent Video Diffusion Models for High-Fidelity Long Video Generationcited by 385
  22. 2019Sliced Score Matching: A Scalable Approach to Density and Score Estimationcited by 381
  23. 2023ResShift: Efficient Diffusion Model for Image Super-resolution by Residual Shiftingcited by 360
  24. 2023Uni-ControlNet: All-in-One Control to Text-to-Image Diffusion Modelscited by 358
  25. 2022DiffusionDet: Diffusion Model for Object Detectioncited by 354
  26. 2023Pix2Video: Video Editing using Image Diffusioncited by 318
  27. 2022Rectified Flow: A Marginal Preserving Approach to Optimal Transportcited by 277
  28. 2022Diffusion Models for Medical Image Analysis: A Comprehensive Surveycited by 269
  29. 2023Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Modelscited by 261
  30. 2022Re-Imagen: Retrieval-Augmented Text-to-Image Generatorcited by 208
  31. 2023A Survey on Video Diffusion Modelscited by 203
  32. 2022Versatile Diffusion: Text, Images and Variations All in One Diffusion Modelcited by 199
  33. 2023UniControl: A Unified Diffusion Model for Controllable Visual Generation In the Wildcited by 197
  34. 2023Mixed-Type Tabular Data Synthesis with Score-based Diffusion in Latent Spacecited by 183
  35. 2024A Survey on Diffusion Models for Inverse Problemscited by 173
  36. 2023Video Probabilistic Diffusion Models in Projected Latent Spacecited by 171
  37. 2023NUWA-XL: Diffusion over Diffusion for eXtremely Long Video Generationcited by 170
  38. 2024ControlNeXt: Powerful and Efficient Control for Image and Video Generationcited by 158

50 most cited of 92 All 92 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 20; 2026: 15.

35 papers.

Sort:

By area:

  1. 2025SVD-LLM V2: Optimizing Singular Value Truncation for Large Language Model Compressioncited by 63
  2. 2025SRPO: Enhancing Multimodal LLM Reasoning via Reflection-Aware Reinforcement Learningcited by 53
  3. 2025MEDA: Dynamic KV Cache Allocation for Efficient Multimodal Long-Context Inferencecited by 31
  4. 2025A Survey of Multimodal Hallucination Evaluation and Detectioncited by 26
  5. 2026QuantVLA: Scale-Calibrated Post-Training Quantization for Vision-Language-Action Modelscited by 25
  6. 2025FPSAttention: Training-Aware FP8 and Sparsity Co-Design for Fast Video Diffusioncited by 19
  7. 2025Enhancing Text-to-Image Diffusion Transformer via Split-Text Conditioningcited by 7
  8. 2025Latent Wavelet Diffusion For Ultra-High-Resolution Image Synthesiscited by 6
  9. 2026FSVideo: Fast Speed Video Diffusion Model in a Highly-Compressed Latent Spacecited by 5
  10. 2025Enhancing Test-Time Scaling of Large Language Models with Hierarchical Retrieval-Augmented MCTScited by 4
  11. 2025DogFit: Domain-guided Fine-tuning for Efficient Transfer Learning of Diffusion Modelscited by 4
  12. 2026Q-Drift: Quantization-Aware Drift Correction for Diffusion Model Samplingcited by 3
23 more
  1. 2025A Comprehensive Review of Diffusion Models in Smart Agriculture: Progress, Applications, and Challengescited by 2
  2. 2025READ: Real-time and Efficient Asynchronous Diffusion for Audio-driven Talking Head Generationcited by 2
  3. 2025Markovian Scale Prediction: A New Era of Visual Autoregressive Generationcited by 2
  4. 2026Token Pruning for In-Context Generation in Diffusion Transformerscited by 2
  5. 2026When Understanding Becomes a Risk: Authenticity and Safety Risks in the Emerging Image Generation Paradigmcited by 2
  6. 2025Generating a Biometrically Unique and Realistic Iris Databasecited by 1
  7. 2025LaTtE-Flow: Layerwise Timestep-Expert Flow-based Transformercited by 1
  8. 2025Boosting Adversarial Transferability via Commonality-Oriented Gradient Optimizationcited by 1
  9. 2025PWD: Prior-Guided and Wavelet-Enhanced Diffusion Model for Limited-Angle CTcited by 1
  10. 2025SlimDiff: Training-Free, Activation-Guided Hands-free Slimming of Diffusion Modelscited by 1
  11. 2025SPID: Distilled Protein Backbone Generationcited by 1
  12. 2025TokenPure: Watermark Removal through Tokenized Appearance and Structural Guidancecited by 1
  13. 2026Al'em do Desempenho: Um Estudo da Confiabilidade de Detectores de Deepfakescited by 1
  14. 2026Order-Optimal Sample Complexity of Rectified Flowscited by 1
  15. 2026Adaptive Visual Autoregressive Acceleration via Dual-Linkage Entropy Analysiscited by 1
  16. 2026Cross-Resolution Distribution Matching for Diffusion Distillationcited by 1
  17. 2026A Unified Measure-Theoretic View of Diffusion, Score-Based, and Flow Matching Generative Modelscited by 1
  18. 2026TRaM-VSR: Importance-Aware Token Routing and Merging for One-Step Diffusion Video Super-Resolutioncited by 1
  19. 2025Why and When Deep is Better than Shallow: Implementation-Agnostic State-Transition Model of Deep Learningcited by 0
  20. 2026Realizing Native INT8 Compute for Diffusion Transformers on Consumer GPUs: A Fused INT8 GEMM Kernel for Ideogram 4.0cited by 0
  21. 2026RE4: Transformation-aware Imitation of Object Interactions Using Manipulation Modescited by 0
  22. 2026Renormalization Group Flow Matching for Scalable Local Generative Modelingcited by 0
  23. 2026ComplexSync: High-Fidelity and Real-Time Lip Sync in Complex Scenarioscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Generative Diffusion Modeling: A Practical Handbook
  2. 2024A Comprehensive Survey on Diffusion Models and Their Applications
  3. 2026Efficient Video Diffusion Models: Advancements and Challenges
  4. 2025Survey of Video Diffusion Models: Foundations, Implementations, and Applications
  5. 2025Diffusion Model Quantization: A Review
  6. 2023State of the Art on Diffusion Models for Visual Computing
  7. 2024Theoretical research on generative diffusion models: an overview
  8. 2023A Survey of Diffusion Based Image Generation Models: Issues and Their Solutions