mlsanityData to 14 Oct 2026

SmoothCache: A Universal Inference Acceleration Technique for Diffusion Transformers

arXiv 2411.10510 ↗First posted 15 Nov 2024cs.LGcited by 21

On the map: Diffusion models → Diffusion foundations and sampling

also in 1 more
Abstract (arXiv)

Diffusion Transformers (DiT) have emerged as powerful generative models for various tasks, including image, video, and speech synthesis. However, their inference process remains computationally expensive due to the repeated evaluation of resource-intensive attention and feed-forward modules. To address this, we introduce SmoothCache, a model-agnostic inference acceleration technique for DiT architectures. SmoothCache leverages the observed high similarity between layer outputs across adjacent diffusion timesteps. By analyzing layer-wise representation errors from a small calibration set, SmoothCache adaptively caches and reuses key features during inference. Our experiments demonstrate that SmoothCache achieves 8% to 71% speed up while maintaining or even improving generation quality across diverse modalities. We showcase its effectiveness on DiT-XL for image generation, Open-Sora for text-to-video, and Stable Audio Open for text-to-audio, highlighting its potential to enable real-time applications and broaden the accessibility of powerful DiT models.

Key facts. Cited by 21 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 33 of its 37 references that are in the map. First posted on arXiv 15 Nov 2024.

Builds on

33 of 37 references.

Closest ancestors · 16

Same direction, newest first

  1. 2024FORA: Fast-Forward Caching in Diffusion Transformer Acceleration1 Jul
  2. 2024Learning-to-Cache: Accelerating Diffusion Transformer via Layer Caching3 Jun
  3. 2024Delta-DiT: A Training-Free Acceleration Method Tailored for Diffusion Transformers3 Jun
  4. 2023Approximate Caching for Efficiently Serving Diffusion Models7 Dec
  5. 2023Cache Me if You Can: Accelerating Diffusion Models through Block Caching6 Dec
  6. 2023DeepCache: Accelerating Diffusion Models for Free1 Dec
  7. 2023On Exact Inversion of DPM-Solvers30 Nov
  8. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis30 Sep
  9. 2023PTQD: Accurate Post-Training Quantization for Diffusion Models18 May
  10. 2022Scalable Diffusion Models with Transformers19 Dec
  11. 2022Post-training Quantization on Diffusion Models28 Nov
  12. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Models2 Nov
4 more
  1. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps2 Jun
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  3. 2020Denoising Diffusion Implicit Models6 Oct
  4. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 17

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  3. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  4. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  5. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  6. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  7. 2022Null-text Inversion for Editing Real Images using Guided Diffusion Modelscited by 1,179
  8. 2024Latte: Latent Diffusion Transformer for Video Generationcited by 546
  9. 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 379
  10. 2023SnapFusion: Text-to-Image Diffusion Model on Mobile Devices within Two Secondscited by 227
  11. 2024Stable Audio Opencited by 197
  12. 2024Real-Time Video Generation with Pyramid Attention Broadcastcited by 149
5 more
  1. 2024VidProM: A Million-scale Real Prompt-Gallery Dataset for Text-to-Video Diffusion Modelscited by 131
  2. 2023DiT-3D: Exploring Plain Diffusion Transformers for 3D Shape Generationcited by 110
  3. 2023The Song Describer Dataset: a Corpus of Audio Captions for Music-and-Language Evaluationcited by 68
  4. 2024Faster Diffusion via Temporal Attention Decompositioncited by 54
  5. 2023ViT-TTS: Visual Text-to-Speech with Scalable Diffusion Transformercited by 24

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 9; 2026: 12.

21 papers.

Sort:

By area:

  1. 2025Accelerating Diffusion Transformer via Gradient-Optimized Cachecited by 15
  2. 2025HiCache: A Plug-in Scaled-Hermite Upgrade for Taylor-Style Cache-then-Forecast Diffusion Accelerationcited by 12
  3. 2025Astraea: A Token-wise Acceleration Framework for Video Diffusion Transformerscited by 11
  4. 2025BWCache: Accelerating Video Diffusion Transformers through Block-Wise Cachingcited by 11
  5. 2026ToProVAR: Efficient Visual Autoregressive Modeling via Tri-Dimensional Entropy-Aware Semantic Analysis and Sparsity Optimizationcited by 5
  6. 2025Generative Diffusion Receivers: Achieving Pilot-Efficient MIMO-OFDM Communicationscited by 3
  7. 2025db-SP: Accelerating Sparse Attention for Visual Generative Models with Dual-Balanced Sequence Parallelismcited by 3
  8. 2025Parallel Diffusion Solver via Residual Dirichlet Policy Optimizationcited by 3
  9. 2025ProCache: Constraint-Aware Feature Caching with Selective Computation for Diffusion Transformer Accelerationcited by 2
  10. 2026Predict to Skip: Linear Multistep Feature Forecasting for Efficient Diffusion Transformerscited by 2
  11. 2025Accelerating Diffusion Transformer-Based Text-to-Speech with Transformer Layer Cachingcited by 0
  12. 2026SoundWeaver: Semantic Warm-Starting for Text-to-Audio Diffusion Servingcited by 0
9 more
  1. 2026Dual-Rate Diffusion: Accelerating diffusion models with an interleaved heavy-light networkcited by 0
  2. 2026Trajectory-Consistent Calibration for Cache-Accelerated Diffusion Modelscited by 0
  3. 2026Kaleido: Algorithm-Hardware Co-Design for Video Diffusion Transformers by Exploiting Latent Space Correlationscited by 0
  4. 2026Deltoris: Enabling Real-time VLA Inference in Embodied AI via Bit-level Sparsity and Speculative Inferencecited by 0
  5. 2026Beyond Attention Masks: Instruction Anchoring for Efficient In-Context Diffusion Generationcited by 0
  6. 2026Spectral-Guided Diffusion: Accelerating Inference via Static Spectral Layer Schedulingcited by 0
  7. 2026GeoShrink: Accelerating Diffusion Transformers with Two Lines of Codecited by 0
  8. 2026ManifoldCache: Training-Free Diffusion Acceleration via Constraint Manifold Cachingcited by 0
  9. 2026Hybrid-Basis Feature Forecasting for Diffusion Sampling Accelerationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025FastCache: Fast Caching for Diffusion Transformer Through Learnable Linear Approximation
  2. 2025OmniCache: A Trajectory-Oriented Global Perspective on Training-Free Cache Reuse for Diffusion Transformer Models
  3. 2026SenCache: Accelerating Diffusion Model Inference via Sensitivity-Aware Caching
  4. 2024HarmoniCa: Harmonizing Training and Inference for Better Feature Caching in Diffusion Transformer Acceleration
  5. 2026Memory-Efficient Training-Free Acceleration of Diffusion Transformers with BaryCache
  6. 2026AdaCorrection: Adaptive Offset Cache Correction for Accurate Diffusion Transformers
  7. 2026DiffSparse: Accelerating Diffusion Transformers with Learned Token Sparsity
  8. 2024Token Caching for Diffusion Transformer Acceleration