mlsanityData to 14 Oct 2026

FastCache: Fast Caching for Diffusion Transformer Through Learnable Linear Approximation

arXiv 2505.20353 ↗First posted 26 May 2025cs.LG cs.AI cs.CVcited by 24

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion Transformers (DiT) are powerful generative models but remain computationally intensive due to their iterative structure and deep transformer stacks. To alleviate this inefficiency, we propose FastCache, a hidden-state-level caching and compression framework that accelerates DiT inference by exploiting redundancy within the model's internal representations. FastCache introduces a dual strategy: (1) a spatial-aware token selection mechanism that adaptively filters redundant tokens based on hidden-state saliency, and (2) a transformer-level cache that reuses latent activations across timesteps when changes fall below a predefined threshold. These modules work jointly to reduce unnecessary computation while preserving generation fidelity through learnable linear approximation. Theoretical analysis shows that FastCache maintains bounded approximation error under a hypothesis-testing-based decision rule. Empirical evaluations across multiple DiT variants demonstrate substantial reductions in latency and memory usage, achieving the best generation quality among existing cache methods, as measured by FID and t-FID. To further improve the speedup of FastCache, we also introduce a token merging module that merges redundant tokens based on k-NN density. Code is available at hrefhttps://github.com/NoakLiu/FastCache-xDiThttps://github.com/NoakLiu/FastCache-xDiT.

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 39 of its 45 references that are in the map. First posted on arXiv 26 May 2025.

Builds on

39 of 45 references.

Closest ancestors · 9

Same direction, newest first

  1. 2025BlockDance: Reuse Structurally Similar Spatio-Temporal Features to Accelerate Diffusion Transformers20 Mar
  2. 2025Accelerating Diffusion Transformer via Error-Optimized Cache31 Jan
  3. 2024LazyDiT: Lazy Learning for the Acceleration of Diffusion Transformers17 Dec
  4. 2024Token Caching for Diffusion Transformer Acceleration27 Sep
  5. 2024Learning-to-Cache: Accelerating Diffusion Transformer via Layer Caching3 Jun
  6. 2024Explaining generative diffusion models via visual analysis for interpretable decision-making process16 Feb
  7. 2023DeepCache: Accelerating Diffusion Models for Free1 Dec
  8. 2023Token Merging for Fast Stable Diffusion30 Mar
  9. 2022Scalable Diffusion Models with Transformers19 Dec

Other references · 30

  1. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  3. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,186
  4. 2023AWQ: Activation-aware Weight Quantization for LLM Compression and Accelerationcited by 1,298
  5. 2022Token Merging: Your ViT But Fastercited by 1,021
  6. 2021DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsificationcited by 864
  7. 2023H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Modelscited by 832
  8. 2023LLM-Pruner: On the Structural Pruning of Large Language Modelscited by 650
  9. 2018Rethinking the Value of Network Pruningcited by 599
  10. 2017BranchyNet: Fast Inference via Early Exiting from Deep Neural Networkscited by 596
  11. 2024Latte: Latent Diffusion Transformer for Video Generationcited by 546
  12. 2024Hybrid LLM: Cost-Efficient and Quality-Aware Query Routingcited by 372
18 more
  1. 2017SkipNet: Learning Dynamic Routing in Convolutional Networkscited by 332
  2. 2022Confident Adaptive Language Modelingcited by 310
  3. 2024Timestep Embedding Tells: It's Time to Cache for Video Diffusion Modelcited by 269
  4. 2019Depth-Adaptive Transformercited by 236
  5. 2020SpAtten: Efficient Sparse Attention Architecture with Cascade Token and Head Pruningcited by 234
  6. 2024Real-Time Video Generation with Pyramid Attention Broadcastcited by 149
  7. 2021IA-RED2: Interpretability-Aware Redundancy Reduction for Vision Transformerscited by 127
  8. 2023Token Fusion: Bridging the Gap between Token Pruning and Token Mergingcited by 125
  9. 2024FasterCache: Training-Free Video Diffusion Model Acceleration with High Qualitycited by 122
  10. 2024Adaptive Caching for Faster Video Generation with Diffusion Transformerscited by 108
  11. 2020Compressing Large-Scale Transformer-Based Models: A Case Study on BERTcited by 96
  12. 2023LooseControl: Lifting ControlNet for Generalized Depth Conditioningcited by 68
  13. 2016AdaNet: Adaptive Structural Learning of Artificial Neural Networkscited by 54
  14. 2023Harnessing the Spatial-Temporal Attention of Diffusion Models for High-Fidelity Text-to-Image Synthesiscited by 51
  15. 2024MT2ST: Adaptive Multi-Task to Single-Task Learningcited by 16
  16. 2024LLMEasyQuant: Scalable Quantization for Parallel and Distributed LLM Inferencecited by 15
  17. 2023AdaMTL: Adaptive Input-dependent Inference for Efficient Multi-Task Learningcited by 5
  18. 2015Feature-based Attention in Convolutional Neural Networkscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 10; 2026: 14.

24 papers.

Sort:

By area:

  1. 2025A Survey on Cache Methods in Diffusion Models: Toward Efficient Multi-Modal Generationcited by 16
  2. 2025ERTACache: Error Rectification and Timesteps Adjustment for Efficient Diffusioncited by 12
  3. 2025BWCache: Accelerating Video Diffusion Transformers through Block-Wise Cachingcited by 11
  4. 2025TinyServe: Query-Aware Cache Selection for Efficient LLM Servingcited by 9
  5. 2026Efficient Video Diffusion Models: Advancements and Challengescited by 7
  6. 2025PiKV: KV Cache Management System for Mixture of Expertscited by 5
  7. 2026Frequency-Aware Error-Bounded Caching for Accelerating Diffusion Transformerscited by 4
  8. 20261.x-Distill: Breaking the Diversity, Quality, and Efficiency Barrier in Distribution Matching Distillationcited by 4
  9. 2025SemToken: Semantic-Aware Tokenization for Efficient Long-Context Language Modelingcited by 3
  10. 2026AdaCorrection: Adaptive Offset Cache Correction for Accurate Diffusion Transformerscited by 3
  11. 2026Accelerating Diffusion-based Video Editing via Heterogeneous Caching: Beyond Full Computing at Sampled Denoising Timestepcited by 3
  12. 2026CachedSearch: Training-Free Cached Exploration for Test-Time Search in Video Diffusioncited by 3
12 more
  1. 2025Plug-and-Play Fidelity Optimization for Diffusion Transformer Acceleration via Cumulative Error Minimizationcited by 2
  2. 2025Anomaly Detection and Early Warning Mechanism for Intelligent Monitoring Systems in Multi-Cloud Environments Based on LLMcited by 1
  3. 2025QuickMerge++: Fast Token Merging with Autoregressive Priorcited by 1
  4. 2026Accelerating Frequency Domain Diffusion Models with Error-Feedback Event-Driven Cachingcited by 1
  5. 2026SAFE-DiT: Semantics-Aware Fast-path Execution for High-Resolution Diffusion Transformerscited by 1
  6. 2026OnlineCache: Learning Dynamic Caching Policies with Error Correction for Efficient Diffusion Inferencecited by 1
  7. 2025Cognitive Load Traces as Symbolic and Visual Accounts of Deep Model Cognitioncited by 0
  8. 2026Rethinking Multi-Condition DiTs: Eliminating Redundant Attention via Position-Alignment and Keyword-Scopingcited by 0
  9. 2026Test-time Sparsity for Extreme Fast Action Diffusioncited by 0
  10. 2026HSMLA: Hierarchical Softmax Multi-scale Linear Attention for Efficient Vision Transformerscited by 0
  11. 2026Magnitude-Direction Decoupling for Fast Video Generation with Flow Matching Modelscited by 0
  12. 2026Koopman Observers for Diffusion Acceleration: Correcting Feature Forecasts with Shallow Measurementscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025ProCache: Constraint-Aware Feature Caching with Selective Computation for Diffusion Transformer Acceleration
  2. 2025OmniCache: A Trajectory-Oriented Global Perspective on Training-Free Cache Reuse for Diffusion Transformer Models
  3. 2026DiffSparse: Accelerating Diffusion Transformers with Learned Token Sparsity
  4. 2024SmoothCache: A Universal Inference Acceleration Technique for Diffusion Transformers
  5. 2024HarmoniCa: Harmonizing Training and Inference for Better Feature Caching in Diffusion Transformer Acceleration
  6. 2024FORA: Fast-Forward Caching in Diffusion Transformer Acceleration
  7. 2026BeCARE: Budgeted Cache Refresh for Diffusion Transformer Acceleration
  8. 2025Accelerating Diffusion Transformer via Increment-Calibrated Caching with Channel-Aware Singular Value Decomposition