mlsanityData to 14 Oct 2026

FORA: Fast-Forward Caching in Diffusion Transformer Acceleration

arXiv 2407.01425 ↗First posted 1 Jul 2024cs.CVcited by 137

On the map: Diffusion models → Diffusion foundations and sampling

Abstract (arXiv)

Diffusion transformers (DiT) have become the de facto choice for generating high-quality images and videos, largely due to their scalability, which enables the construction of larger models for enhanced performance. However, the increased size of these models leads to higher inference costs, making them less attractive for real-time applications. We present Fast-FORward CAching (FORA), a simple yet effective approach designed to accelerate DiT by exploiting the repetitive nature of the diffusion process. FORA implements a caching mechanism that stores and reuses intermediate outputs from the attention and MLP layers across denoising steps, thereby reducing computational overhead. This approach does not require model retraining and seamlessly integrates with existing transformer-based diffusion models. Experiments show that FORA can speed up diffusion transformers several times over while only minimally affecting performance metrics such as the IS Score and FID. By enabling faster processing with minimal trade-offs in quality, FORA represents a significant advancement in deploying diffusion transformers for real-time applications. Code will be made publicly available at: https://github.com/prathebaselva/FORA.

Key facts. Cited by 137 papers in the map (data through 14 Oct 2026). Placed in Diffusion foundations and sampling, one of 14 directions in Diffusion models. Builds on 30 of its 33 references that are in the map. First posted on arXiv 1 Jul 2024.

Builds on

30 of 33 references.

Closest ancestors · 20

Same direction, newest first

  1. 2024Attention-Driven Training-Free Efficiency Enhancement of Diffusion Models8 May
  2. 2024PixArt-Sigma: Weak-to-Strong Training of Diffusion Transformer for 4K Text-to-Image Generation7 Mar
  3. 2024PIXART-δ: Fast and Controllable Image Generation with Latent Consistency Models10 Jan
  4. 2023Cache Me if You Can: Accelerating Diffusion Models through Block Caching6 Dec
  5. 2023DeepCache: Accelerating Diffusion Models for Free1 Dec
  6. 2023DPM-Solver-v3: Improved Diffusion ODE Solver with Empirical Model Statistics20 Oct
  7. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis30 Sep
  8. 2023Structural Pruning for Diffusion Models18 May
  9. 2023PTQD: Accurate Post-Training Quantization for Diffusion Models18 May
  10. 2023Consistency Models2 Mar
  11. 2022Scalable Diffusion Models with Transformers19 Dec
  12. 2022Post-training Quantization on Diffusion Models28 Nov
8 more
  1. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Steps2 Jun
  2. 2022Pseudo Numerical Methods for Diffusion Models on Manifolds20 Feb
  3. 2022Analytic-DPM: an Analytic Estimate of the Optimal Reverse Variance in Diffusion Probabilistic Models17 Jan
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Models20 Dec
  5. 2021Cascaded Diffusion Models for High Fidelity Image Generation30 May
  6. 2021Diffusion Models Beat GANs on Image Synthesis11 May
  7. 2020Denoising Diffusion Implicit Models6 Oct
  8. 2020Denoising Diffusion Probabilistic Models19 Jun

Other references · 10

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  6. 2022Video Diffusion Modelscited by 2,305
  7. 2023Scaling up GANs for Text-to-Image Synthesiscited by 492
  8. 2024Faster Diffusion via Temporal Attention Decompositioncited by 54
  9. 2023DREAM: Diffusion Rectification and Estimation-Adaptive Modelscited by 12
  10. 2023Accelerating Text-to-Image Editing via Cache-Enabled Sparse Diffusion Inferencecited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 9; 2025: 53; 2026: 75.

137 papers.

Sort:

By area:

  1. 2024Timestep Embedding Tells: It's Time to Cache for Video Diffusion Modelcited by 269
  2. 2025dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Cachingcited by 169
  3. 2024Accelerating Diffusion Transformers with Token-wise Feature Cachingcited by 156
  4. 2025From Reusing to Forecasting: Accelerating Diffusion Models with TaylorSeerscited by 136
  5. 2024Adaptive Caching for Faster Video Generation with Diffusion Transformerscited by 108
  6. 2025EfficientVLA: Training-Free Acceleration and Compression for Vision-Language-Action Modelscited by 93
  7. 2025OminiControl2: Efficient Conditioning for Diffusion Transformerscited by 57
  8. 2025EEdit: Rethinking the Spatial and Temporal Redundancy for Efficient Image Editingcited by 45
  9. 2025Less is Enough: Training-Free Video Diffusion Acceleration via Runtime-Adaptive Cachingcited by 45
  10. 2025SpecPrune-VLA: Accelerating Vision-Language-Action Models via Action-Aware Self-Speculative Pruningcited by 43
  11. 2025Follow-Your-Emoji-Faster: Towards Efficient, Fine-Controllable, and Expressive Freestyle Portrait Animationcited by 43
  12. 2025FlashVSR: Towards Real-Time Diffusion-Based Streaming Video Super-Resolutioncited by 43
38 more
  1. 2025DiCache: Let Diffusion Model Determine Its Own Cachecited by 42
  2. 2025MagCache: Fast Video Generation with Magnitude-Aware Cachecited by 41
  3. 2025Efficient Diffusion Models: A Surveycited by 35
  4. 2024Token Caching for Diffusion Transformer Accelerationcited by 30
  5. 2025StreamDiffusionV2: A Streaming System for Dynamic and Interactive Video Generationcited by 30
  6. 2025SpeCa: Accelerating Diffusion Transformers with Speculative Feature Cachingcited by 29
  7. 2024Rethinking Token-wise Feature Caching: Accelerating Diffusion Transformers with Dual Feature Cachingcited by 28
  8. 2026Flow caching for autoregressive video generationcited by 25
  9. 2024SmoothCache: A Universal Inference Acceleration Technique for Diffusion Transformerscited by 21
  10. 2025DiTFastAttnV2: Head-wise Attention Compression for Multi-Modality Diffusion Transformerscited by 20
  11. 2025FreqCa: Accelerating Diffusion Models via Frequency-Aware Cachingcited by 20
  12. 2025Compute Only 16 Tokens in One Timestep: Accelerating Diffusion Transformers with Cluster-Driven Feature Cachingcited by 19
  13. 2025FullDiT2: Efficient In-Context Conditioning for Video Diffusion Transformerscited by 18
  14. 2025OmniCache: A Trajectory-Oriented Global Perspective on Training-Free Cache Reuse for Diffusion Transformer Modelscited by 18
  15. 2024HarmoniCa: Harmonizing Training and Inference for Better Feature Caching in Diffusion Transformer Accelerationcited by 17
  16. 2025CAT Pruning: Cluster-Aware Token Pruning For Text-to-Image Diffusion Modelscited by 16
  17. 2025A Survey on Cache Methods in Diffusion Models: Toward Efficient Multi-Modal Generationcited by 16
  18. 2025Accelerating Diffusion Transformer via Gradient-Optimized Cachecited by 15
  19. 2025CacheQuant: Comprehensively Accelerated Diffusion Modelscited by 13
  20. 2024Towards Stabilized and Efficient Diffusion Transformers through Long-Skip-Connections with Spectral Constraintscited by 12
  21. 2025Accelerating Diffusion Transformer via Increment-Calibrated Caching with Channel-Aware Singular Value Decompositioncited by 12
  22. 2025HiCache: A Plug-in Scaled-Hermite Upgrade for Taylor-Style Cache-then-Forecast Diffusion Accelerationcited by 12
  23. 2025BWCache: Accelerating Video Diffusion Transformers through Block-Wise Cachingcited by 11
  24. 2026SeaCache: Spectral-Evolution-Aware Cache for Accelerating Diffusion Modelscited by 11
  25. 2025Let Features Decide Their Own Solvers: Hybrid Feature Caching for Diffusion Transformerscited by 10
  26. 2025LeMiCa: Lexicographic Minimax Path Caching for Efficient Diffusion-Based Video Generationcited by 10
  27. 2026MeanCache: From Instantaneous to Average Velocity for Accelerating Flow Matching Inferencecited by 10
  28. 2026TC-Pad'e: Trajectory-Consistent Pad'e Approximation for Diffusion Accelerationcited by 10
  29. 2026Elastic Diffusion Transformercited by 8
  30. 2026Denoising as Path Planning: Training-Free Acceleration of Diffusion Models with DPCachecited by 8
  31. 2026PreciseCache: Precise Feature Caching for Efficient and High-fidelity Video Generationcited by 8
  32. 2025SpotEdit: Selective Region Editing in Diffusion Transformerscited by 7
  33. 2026DisCa: Accelerating Video Diffusion Transformers with Distillation-Compatible Learnable Feature Cachingcited by 7
  34. 2026SenCache: Accelerating Diffusion Model Inference via Sensitivity-Aware Cachingcited by 7
  35. 2026Adaptive Spectral Feature Forecasting for Diffusion Sampling Accelerationcited by 7
  36. 2025Foresight: Adaptive Layer Reuse for Accelerated and High-Quality Text-to-Video Generationcited by 6
  37. 2025Forecasting When to Forecast: Accelerating Diffusion Models with Confidence-Gated Taylorcited by 6
  38. 2025SuperGen: An Efficient Ultra-high-resolution Video Generation System with Sketching and Tilingcited by 6

50 most cited of 137 All 137 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Learning-to-Cache: Accelerating Diffusion Transformer via Layer Caching
  2. 2025Accelerating Diffusion Transformer via Error-Optimized Cache
  3. 2025FastCache: Fast Caching for Diffusion Transformer Through Learnable Linear Approximation
  4. 2026Frequency-Aware Error-Bounded Caching for Accelerating Diffusion Transformers
  5. 2024Delta-DiT: A Training-Free Acceleration Method Tailored for Diffusion Transformers
  6. 2026BeCARE: Budgeted Cache Refresh for Diffusion Transformer Acceleration
  7. 2025ProCache: Constraint-Aware Feature Caching with Selective Computation for Diffusion Transformer Acceleration
  8. 2026CacheScope: Exact-Compute Allocation for Training-Free Diffusion Acceleration