mlsanityData to 14 Oct 2026

Cache Me if You Can: Accelerating Diffusion Models through Block Caching

arXiv 2312.03209 ↗First posted 6 Dec 2023cs.CVcited by 151

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion models have recently revolutionized the field of image synthesis due to their ability to generate photorealistic images. However, one of the major drawbacks of diffusion models is that the image generation process is costly. A large image-to-image network has to be applied many times to iteratively refine an image from random noise. While many recent works propose techniques to reduce the number of required steps, they generally treat the underlying denoising network as a black box. In this work, we investigate the behavior of the layers within the network and find that 1) the layers' output changes smoothly over time, 2) the layers show distinct patterns of change, and 3) the change from step to step is often very small. We hypothesize that many layer computations in the denoising network are redundant. Leveraging this, we introduce block caching, in which we reuse outputs from layer blocks of previous steps to speed up inference. Furthermore, we propose a technique to automatically determine caching schedules based on each block's changes over timesteps. In our experiments, we show through FID, human evaluation and qualitative analysis that Block Caching allows to generate images with higher visual quality at the same computational cost. We demonstrate this for different state-of-the-art models (LDM and EMU) and solvers (DDIM and DPM).

Key facts. Cited by 151 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 55 of its 60 references that are in the map. First posted on arXiv 6 Dec 2023.

Builds on

55 of 60 references.

Closest ancestors · 24

Same direction, newest first

  1. 2023Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference6 Oct
  2. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis30 Sep
  3. 2023Improved Order Analysis and Design of Exponential Integrator for Diffusion Models Sampling4 Aug
  4. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis4 Jul
  5. 2023Optimal Linear Subspace Search: Learning to Construct Fast and High-Quality Schedulers for Diffusion Models24 May
  6. 2023Consistency Models2 Mar
  7. 2023UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Models9 Feb
  8. 2022Diffusion Probabilistic Model Made Slim27 Nov
  9. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models2 Nov
  10. 2022GENIE: Higher-Order Denoising Diffusion Solvers11 Oct
  11. 2022On Distillation of Guided Diffusion Models6 Oct
  12. 2022Classifier-Free Diffusion Guidance26 Jul
12 more
  1. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps2 Jun
  2. 2022Elucidating the Design Space of Diffusion-Based Generative Models1 Jun
  3. 2022Fast Sampling of Diffusion Models with Exponential Integrator29 Apr
  4. 2022Pseudo Numerical Methods for Diffusion Models on Manifolds20 Feb
  5. 2022Learning Fast Samplers for Diffusion Models by Differentiating Through Sample Quality11 Feb
  6. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  7. 2021Bilateral Denoising Diffusion Models26 Aug
  8. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  9. 2021Knowledge Distillation in Iterative Generative Models for Improved Sampling Speed7 Jan
  10. 2020Score-Based Generative Modeling through Stochastic Differential Equations26 Nov
  11. 2020Denoising Diffusion Implicit Models6 Oct
  12. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 31

  1. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  2. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  3. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  4. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  6. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  7. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  8. 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,253
  9. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  10. 2018Large Scale GAN Training for High Fidelity Natural Image Synthesiscited by 2,626
  11. 2022InstructPix2Pix: Learning to Follow Image Editing Instructionscited by 2,611
  12. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,431
19 more
  1. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,742
  2. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,387
  3. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,295
  4. 2022MaskGIT: Masked Generative Image Transformercited by 1,099
  5. 2018Self-Attention Generative Adversarial Networkscited by 809
  6. 2016Generative Adversarial Text to Image Synthesiscited by 735
  7. 2017AttnGAN: Fine-Grained Text to Image Generation with Attentional Generative Adversarial Networkscited by 664
  8. 2023GAIA-1: A Generative World Model for Autonomous Drivingcited by 637
  9. 2022Compositional Visual Generation with Composable Diffusion Modelscited by 624
  10. 2023Scaling up GANs for Text-to-Image Synthesiscited by 492
  11. 2019DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-to-Image Synthesiscited by 308
  12. 2023Emu: Enhancing Image Generation Models Using Photogenic Needles in a Haystackcited by 279
  13. 2019Controllable Text-to-Image Generationcited by 187
  14. 2019MirrorGAN: Learning Text-to-image Generation by Redescriptioncited by 176
  15. 2020DF-GAN: A Simple and Effective Baseline for Text-to-Image Synthesiscited by 162
  16. 2023RAPHAEL: Text-to-Image Generation via Large Mixture of Diffusion Pathscited by 142
  17. 2022ERNIE-ViLG 2.0: Improving Text-to-Image Diffusion Model with Knowledge-Enhanced Mixture-of-Denoising-Expertscited by 112
  18. 2016Identity Matters in Deep Learningcited by 109
  19. 2023Bespoke Solvers for Generative Flow Modelscited by 37

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 38; 2025: 61; 2026: 51.

151 papers.

Sort:

By area:

  1. 2024Timestep Embedding Tells: It's Time to Cache for Video Diffusion Modelcited by 269
  2. 2024Accelerating Diffusion Transformers with Token-wise Feature Cachingcited by 156
  3. 2024Real-Time Video Generation with Pyramid Attention Broadcastcited by 149
  4. 2025dKV-Cache: The Cache for Diffusion Language Modelscited by 143
  5. 2024Learning-to-Cache: Accelerating Diffusion Transformer via Layer Cachingcited by 142
  6. 2024FORA: Fast-Forward Caching in Diffusion Transformer Accelerationcited by 137
  7. 2024FasterCache: Training-Free Video Diffusion Model Acceleration with High Qualitycited by 122
  8. 2024Adaptive Caching for Faster Video Generation with Diffusion Transformerscited by 108
  9. 2025SP-VLA: A Joint Model Scheduling and Token Pruning Approach for VLA Model Accelerationcited by 58
  10. 2024Faster Diffusion via Temporal Attention Decompositioncited by 54
  11. 2025Less is Enough: Training-Free Video Diffusion Acceleration via Runtime-Adaptive Cachingcited by 45
  12. 2025MagCache: Fast Video Generation with Magnitude-Aware Cachecited by 41
38 more
  1. 2024Looking Backward: Streaming Video-to-Video Translation with Feature Bankscited by 38
  2. 2024Imagine Flash: Accelerating Emu Diffusion Models with Backward Distillationcited by 37
  3. 2024LazyDiT: Lazy Learning for the Acceleration of Diffusion Transformerscited by 37
  4. 2025Efficient Diffusion Models: A Surveycited by 35
  5. 2023Faster Diffusion: Rethinking the Role of the Encoder for Diffusion Model Inferencecited by 34
  6. 2025Training-Free Efficient Video Generation via Dynamic Token Carvingcited by 34
  7. 2024T-Stitch: Accelerating Sampling in Pre-Trained Diffusion Models with Trajectory Stitchingcited by 33
  8. 2024Collaborative Decoding Makes Visual Auto-Regressive Modeling Efficientcited by 33
  9. 2024SlimFlow: Training Smaller One-Step Diffusion Models with Rectified Flowcited by 30
  10. 2025StreamDiffusionV2: A Streaming System for Dynamic and Interactive Video Generationcited by 30
  11. 2025Forecast then Calibrate: Feature Caching as ODE for Efficient Diffusion Transformerscited by 26
  12. 2025BlockDance: Reuse Structurally Similar Spatio-Temporal Features to Accelerate Diffusion Transformerscited by 24
  13. 2024AsyncDiff: Parallelizing Diffusion Models by Asynchronous Denoisingcited by 22
  14. 2024Importance-Based Token Merging for Efficient Image and Video Generationcited by 22
  15. 2025Memory-Efficient Visual Autoregressive Modeling with Scale-Aware KV Cache Compressioncited by 22
  16. 2024SmoothCache: A Universal Inference Acceleration Technique for Diffusion Transformerscited by 21
  17. 2025TREAD: Token Routing for Efficient Architecture-agnostic Diffusion Trainingcited by 21
  18. 2025DiTFastAttnV2: Head-wise Attention Compression for Multi-Modality Diffusion Transformerscited by 20
  19. 2024Hash3D: Training-free Acceleration for 3D Generationcited by 19
  20. 2025OmniCache: A Trajectory-Oriented Global Perspective on Training-Free Cache Reuse for Diffusion Transformer Modelscited by 18
  21. 2024HarmoniCa: Harmonizing Training and Inference for Better Feature Caching in Diffusion Transformer Accelerationcited by 17
  22. 2025Attend to Not Attended: Structure-then-Detail Token Merging for Post-training DiT Accelerationcited by 17
  23. 2024Presto! Distilling Steps and Layers for Accelerating Music Generationcited by 16
  24. 2024AsymRnR: Video Diffusion Transformers Acceleration with Asymmetric Reduction and Restorationcited by 16
  25. 2025Cached Adaptive Token Merging: Dynamic Token Reduction and Redundant Computation Elimination in Diffusion Modelcited by 16
  26. 2025CAT Pruning: Cluster-Aware Token Pruning For Text-to-Image Diffusion Modelscited by 16
  27. 2025A Survey on Cache Methods in Diffusion Models: Toward Efficient Multi-Modal Generationcited by 16
  28. 2025Principled RL for Flow Matching Emerges from the Chunk-level Policy Optimizationcited by 14
  29. 2024Not All Prompts Are Made Equal: Prompt-based Pruning of Text-to-Image Diffusion Modelscited by 13
  30. 2024A Simple Early Exiting Framework for Accelerated Sampling in Diffusion Modelscited by 13
  31. 2025CacheQuant: Comprehensively Accelerated Diffusion Modelscited by 13
  32. 2024Invertible Diffusion Models for Compressed Sensingcited by 12
  33. 2024Towards Stabilized and Efficient Diffusion Transformers through Long-Skip-Connections with Spectral Constraintscited by 12
  34. 2025FlexiDiT: Your Diffusion Transformer Can Easily Generate High-Quality Samples with Less Computecited by 12
  35. 2025Accelerating Diffusion Transformer via Increment-Calibrated Caching with Channel-Aware Singular Value Decompositioncited by 12
  36. 2025DIFFVSGG: Diffusion-Driven Online Video Scene Graph Generationcited by 11
  37. 2025Scale-wise Distillation of Diffusion Modelscited by 11
  38. 2025Astraea: A Token-wise Acceleration Framework for Video Diffusion Transformerscited by 11

50 most cited of 151 All 151 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025AB-Cache: Training-Free Acceleration of Diffusion Models via Adams-Bashforth Cached Feature Reuse
  2. 2023DeepCache: Accelerating Diffusion Models for Free
  3. 2026SenCache: Accelerating Diffusion Model Inference via Sensitivity-Aware Caching
  4. 2026ReCache: Learning Budget-Aware Caching Schedules for Diffusion Models via REINFORCE
  5. 2026Fixed State, Long Reach: What a Constant-Size Cache Buys Block Diffusion at Scale
  6. 2025No Cache Left Idle: Accelerating diffusion model via Extreme-slimming Caching
  7. 2026ACID: Adaptive Caching for vIDeo generation
  8. 2026Budget-Constrained Step-Level Diffusion Caching