mlsanityData to 14 Oct 2026

Timestep Embedding Tells: It's Time to Cache for Video Diffusion Model

arXiv 2411.19108 ↗First posted 28 Nov 2024cs.CVcited by 269

On the map: Diffusion models → Video generation

Abstract (arXiv)

As a fundamental backbone for video generation, diffusion models are challenged by low inference speed due to the sequential nature of denoising. Previous methods speed up the models by caching and reusing model outputs at uniformly selected timesteps. However, such a strategy neglects the fact that differences among model outputs are not uniform across timesteps, which hinders selecting the appropriate model outputs to cache, leading to a poor balance between inference efficiency and visual quality. In this study, we introduce Timestep Embedding Aware Cache (TeaCache), a training-free caching approach that estimates and leverages the fluctuating differences among model outputs across timesteps. Rather than directly using the time-consuming model outputs, TeaCache focuses on model inputs, which have a strong correlation with the modeloutputs while incurring negligible computational cost. TeaCache first modulates the noisy inputs using the timestep embeddings to ensure their differences better approximating those of model outputs. TeaCache then introduces a rescaling strategy to refine the estimated differences and utilizes them to indicate output caching. Experiments show that TeaCache achieves up to 4.41x acceleration over Open-Sora-Plan with negligible (-0.07% Vbench score) degradation of visual quality.

Key facts. Cited by 269 papers in the map (data through 14 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 48 of its 58 references that are in the map. First posted on arXiv 28 Nov 2024.

Builds on

48 of 58 references.

Closest ancestors · 11

Same direction, newest first

  1. 2024Adaptive Caching for Faster Video Generation with Diffusion Transformers4 Nov
  2. 2024FasterCache: Training-Free Video Diffusion Model Acceleration with High Quality25 Oct
  3. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer12 Aug
  4. 2024VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Models17 Jan
  5. 2024Latte: Latent Diffusion Transformer for Video Generation5 Jan
  6. 2023VideoLCM: Video Latent Consistency Model14 Dec
  7. 2023DreamVideo: Composing Your Dream Videos with Customized Subject and Motion7 Dec
  8. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets25 Nov
  9. 2023VideoCrafter1: Open Diffusion Models for High-Quality Video Generation30 Oct
  10. 2023ModelScope Text-to-Video Technical Report12 Aug
  11. 2022Video Diffusion Models7 Apr

Other references · 37

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  5. 2020Denoising Diffusion Implicit Modelscited by 9,781
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  7. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,058
  8. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  9. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  10. 2022Scalable Diffusion Models with Transformerscited by 6,054
  11. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  12. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,673
25 more
  1. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 3,045
  2. 2022DPM-Solver: A Fast ODE Solver for Diffusion Probabilistic Model Sampling in Around 10 Stepscited by 1,979
  3. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
  4. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Modelscited by 938
  5. 2023Adversarial Diffusion Distillationcited by 759
  6. 2022On Distillation of Guided Diffusion Modelscited by 725
  7. 2022Post-training Quantization on Diffusion Modelscited by 267
  8. 2023SnapFusion: Text-to-Image Diffusion Model on Mobile Devices within Two Secondscited by 227
  9. 2023Token Merging for Fast Stable Diffusioncited by 224
  10. 2021Gotta Go Fast When Generating Data with Score-Based Modelscited by 219
  11. 2024T2V-CompBench: A Comprehensive Benchmark for Compositional Text-to-video Generationcited by 185
  12. 2024Delta-DiT: A Training-Free Acceleration Method Tailored for Diffusion Transformerscited by 175
  13. 2023PTQD: Accurate Post-Training Quantization for Diffusion Modelscited by 170
  14. 2023Cache Me if You Can: Accelerating Diffusion Models through Block Cachingcited by 151
  15. 2024Real-Time Video Generation with Pyramid Attention Broadcastcited by 149
  16. 2024Learning-to-Cache: Accelerating Diffusion Transformer via Layer Cachingcited by 142
  17. 2024FORA: Fast-Forward Caching in Diffusion Transformer Accelerationcited by 137
  18. 2023Parallel Sampling of Diffusion Modelscited by 112
  19. 2024DistriFusion: Distributed Parallel Inference for High-Resolution Diffusion Modelscited by 98
  20. 2024Q-DiT: Accurate Post-Training Quantization for Diffusion Transformerscited by 94
  21. 2023Temporal Dynamic Quantization for Diffusion Modelscited by 81
  22. 2017DeepCache: Principled Cache for Mobile Deep Visioncited by 57
  23. 2024DreamVideo-2: Zero-Shot Subject-Driven Video Customization with Precise Motion Controlcited by 43
  24. 2024Attention-Driven Training-Free Efficiency Enhancement of Diffusion Modelscited by 38
  25. 2023FRDiff : Feature Reuse for Universal Training-free Acceleration of Diffusion Modelscited by 20

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 90; 2026: 179.

269 papers.

Sort:

By area:

  1. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  2. 2026World Action Models are Zero-shot Policiescited by 465
  3. 2025SkyReels-V2: Infinite-length Film Generative Modelcited by 229
  4. 2025Sparse VideoGen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsitycited by 217
  5. 2025Frame Context Packing and Drift Prevention in Next-Frame-Prediction Video Diffusion Modelscited by 143
  6. 2025From Reusing to Forecasting: Accelerating Diffusion Models with TaylorSeerscited by 136
  7. 2025Sparse VideoGen2: Accelerate Video Generation with Sparse Attention via Semantic-Aware Permutationcited by 106
  8. 2025Lumina-Image 2.0: A Unified and Efficient Image Generative Frameworkcited by 96
  9. 2025Autoregressive Adversarial Post-Training for Real-Time Interactive Video Generationcited by 90
  10. 2025Yume: An Interactive World Generation Modelcited by 81
  11. 2025Radial Attention: O(nlog n) Sparse Attention with Energy Decay for Long Video Generationcited by 69
  12. 2026Helios: Real Real-Time Long Video Generation Modelcited by 65
38 more
  1. 2025Less is Enough: Training-Free Video Diffusion Acceleration via Runtime-Adaptive Cachingcited by 45
  2. 2025FlashVSR: Towards Real-Time Diffusion-Based Streaming Video Super-Resolutioncited by 43
  3. 2025DiCache: Let Diffusion Model Determine Its Own Cachecited by 42
  4. 2025MagCache: Fast Video Generation with Magnitude-Aware Cachecited by 41
  5. 2026DreamX-World 1.0: A General-Purpose Interactive World Modelcited by 40
  6. 2025VMoBA: Mixture-of-Block Attention for Video Diffusion Modelscited by 37
  7. 2025Training-Free Efficient Video Generation via Dynamic Token Carvingcited by 34
  8. 2025Sparse-vDiT: Unleashing the Power of Sparse Attention to Accelerate Video Diffusion Transformerscited by 34
  9. 2025StreamDiffusionV2: A Streaming System for Dynamic and Interactive Video Generationcited by 30
  10. 2025EchoMimicV3: 1.3B Parameters are All You Need for Unified Multi-Modal and Multi-Task Human Animationcited by 29
  11. 2025SpeCa: Accelerating Diffusion Transformers with Speculative Feature Cachingcited by 29
  12. 2026Flow caching for autoregressive video generationcited by 25
  13. 2025FastCache: Fast Caching for Diffusion Transformer Through Learnable Linear Approximationcited by 24
  14. 2025DreamRelation: Relation-Centric Video Customizationcited by 23
  15. 2025VORTA: Efficient Video Diffusion via Routing Sparse Attentioncited by 23
  16. 2026Efficient Autoregressive Video Diffusion with Dummy Headcited by 20
  17. 2025Compute Only 16 Tokens in One Timestep: Accelerating Diffusion Transformers with Cluster-Driven Feature Cachingcited by 19
  18. 2026Dream-Tac: A Unified Tactile World Action Model for Contact-Rich Robot Manipulationcited by 19
  19. 2025OmniCache: A Trajectory-Oriented Global Perspective on Training-Free Cache Reuse for Diffusion Transformer Modelscited by 18
  20. 2025A Survey on Cache Methods in Diffusion Models: Toward Efficient Multi-Modal Generationcited by 16
  21. 2025Model Reveals What to Cache: Profiling-Based Feature Reuse for Video Diffusion Modelscited by 15
  22. 2025DVD-Quant: Data-free Video Diffusion Transformers Quantizationcited by 15
  23. 2025DriveGen3D: Boosting Feed-Forward Driving Scene Generation with Efficient Video Diffusioncited by 15
  24. 2026Your One-Stop Solution for AI-Generated Video Detectioncited by 15
  25. 2026Forcing-KV: Hybrid KV Cache Compression for Efficient Autoregressive Video Diffusion Modelscited by 15
  26. 2025SkyReels-Audio: Omni Audio-Conditioned Talking Portraits in Video Diffusion Transformerscited by 14
  27. 2025Principled RL for Flow Matching Emerges from the Chunk-level Policy Optimizationcited by 14
  28. 2025Motion-Aware Generative Frame Interpolationcited by 13
  29. 2025AB-Cache: Training-Free Acceleration of Diffusion Models via Adams-Bashforth Cached Feature Reusecited by 13
  30. 2025S2Q-VDiT: Accurate Quantized Video Diffusion Transformer with Salient Data and Sparse Token Distillationcited by 13
  31. 2025Block-wise Adaptive Caching for Accelerating Diffusion Policycited by 12
  32. 2025ERTACache: Error Rectification and Timesteps Adjustment for Efficient Diffusioncited by 12
  33. 2026Light Forcing: Accelerating Autoregressive Video Diffusion via Sparse Attentioncited by 12
  34. 2025Astraea: A Token-wise Acceleration Framework for Video Diffusion Transformerscited by 11
  35. 2025SADA: Stability-guided Adaptive Diffusion Accelerationcited by 11
  36. 2025BWCache: Accelerating Video Diffusion Transformers through Block-Wise Cachingcited by 11
  37. 2026SeaCache: Spectral-Evolution-Aware Cache for Accelerating Diffusion Modelscited by 11
  38. 2025Re-ttention: Ultra Sparse Visual Generation via Attention Statistical Reshapecited by 10

50 most cited of 269 All 269 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Rolling Sink: Bridging Limited-Horizon Training and Open-Ended Testing in Autoregressive Video Diffusion
  2. 2023DeepCache: Accelerating Diffusion Models for Free
  3. 2025No Cache Left Idle: Accelerating diffusion model via Extreme-slimming Caching
  4. 2024Redefining Temporal Modeling in Video Diffusion: The Vectorized Timestep Approach
  5. 2025FreqCa: Accelerating Diffusion Models via Frequency-Aware Caching
  6. 2026Test-Time Scaling for Video Diffusion Models via Diagnosis-Guided Candidate Recycling
  7. 2026Efficient Video Diffusion Models: Advancements and Challenges
  8. 2025Adaptive Hybrid Caching for Efficient Text-to-Video Diffusion Model Acceleration