mlsanityData to 7 Oct 2026

Faster Diffusion via Temporal Attention Decomposition

arXiv 2404.02747 ↗First posted 3 Apr 2024cs.CVcited by 54

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

We explore the role of attention mechanism during inference in text-conditional diffusion models. Empirical observations suggest that cross-attention outputs converge to a fixed point after several inference steps. The convergence time naturally divides the entire inference process into two phases: an initial phase for planning text-oriented visual semantics, which are then translated into images in a subsequent fidelity-improving phase. Cross-attention is essential in the initial phase but almost irrelevant thereafter. However, self-attention initially plays a minor role but becomes crucial in the second phase. These findings yield a simple and training-free method known as temporally gating the attention (TGATE), which efficiently generates images by caching and reusing attention outputs at scheduled time steps. Experimental results show when widely applied to various existing text-conditional diffusion models, TGATE accelerates these models by 10%-50%. The code of TGATE is available at https://github.com/HaozheLiu-ST/T-GATE.

Key facts. Cited by 54 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 43 of its 59 references that are in the map. First posted on arXiv 3 Apr 2024.

Builds on

43 of 59 references.

Closest ancestors · 7

Same direction, newest first

  1. 2024ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignment8 Mar
  2. 2023BoxDiff: Text-to-Image Synthesis with Training-Free Box-Constrained Diffusion20 Jul
  3. 2023SnapFusion: Text-to-Image Diffusion Model on Mobile Devices within Two Seconds1 Jun
  4. 2023Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models31 Jan
  5. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers2 Nov
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr

Other references · 36

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  5. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  6. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  7. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  8. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  9. 2020Denoising Diffusion Implicit Modelscited by 9,655
  10. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  11. 2022Scalable Diffusion Models with Transformerscited by 5,931
  12. 2022Classifier-Free Diffusion Guidancecited by 5,717
24 more
  1. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  2. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  3. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  4. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
  5. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  6. 2022Prompt-to-Prompt Image Editing with Cross Attention Controlcited by 2,409
  7. 2023Consistency Modelscited by 1,778
  8. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Modelscited by 924
  9. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
  10. 2023Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inferencecited by 826
  11. 2021Linear Transformers Are Secretly Fast Weight Programmerscited by 471
  12. 2023DeepCache: Accelerating Diffusion Models for Freecited by 346
  13. 2024Long-CLIP: Unlocking the Long-Text Capability of CLIPcited by 298
  14. 2015Highway Networkscited by 257
  15. 2020Understanding the Role of Individual Units in a Deep Neural Networkcited by 254
  16. 2023Token Merging for Fast Stable Diffusioncited by 220
  17. 2022mPLUG: Effective and Efficient Vision-Language Learning by Cross-modal Skip-connectionscited by 185
  18. 2023Cache Me if You Can: Accelerating Diffusion Models through Block Cachingcited by 149
  19. 2023CADS: Unleashing the Diversity of Diffusion Models through Condition-Annealed Samplingcited by 124
  20. 2022Diffusion Probabilistic Model Made Slimcited by 121
  21. 2024Progressive Knowledge Distillation Of Stable Diffusion XL Using Layer Level Losscited by 51
  22. 2024T-Stitch: Accelerating Sampling in Pre-Trained Diffusion Models with Trajectory Stitchingcited by 32
  23. 2023Adaptive Guidance: Training-free Acceleration of Conditional Diffusion Modelscited by 29
  24. 2023Learning to Identify Critical States for Reinforcement Learning from Videoscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 13; 2025: 26; 2026: 15.

54 papers.

Sort:

By area:

  1. 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 401
  2. 2024Delta-DiT: A Training-Free Acceleration Method Tailored for Diffusion Transformerscited by 169
  3. 2025dKV-Cache: The Cache for Diffusion Language Modelscited by 140
  4. 2024FORA: Fast-Forward Caching in Diffusion Transformer Accelerationcited by 127
  5. 2025Training-free and Adaptive Sparse Attention for Efficient Long Video Generationcited by 65
  6. 2024MarDini: Masked Autoregressive Diffusion for Video Generation at Scalecited by 49
  7. 2024Towards Understanding the Working Mechanism of Text-to-Image Diffusion Modelcited by 48
  8. 2025EEdit: Rethinking the Spatial and Temporal Redundancy for Efficient Image Editingcited by 45
  9. 2025Follow-Your-Emoji-Faster: Towards Efficient, Fine-Controllable, and Expressive Freestyle Portrait Animationcited by 41
  10. 2025Less is Enough: Training-Free Video Diffusion Acceleration via Runtime-Adaptive Cachingcited by 39
  11. 2025FastVAR: Linear Visual Autoregressive Modeling via Cached Token Pruningcited by 34
  12. 2024Learning Flow Fields in Attention for Controllable Person Image Generationcited by 27
38 more
  1. 2026Transition Matching Distillation for Fast Video Generationcited by 27
  2. 2024AsyncDiff: Parallelizing Diffusion Models by Asynchronous Denoisingcited by 22
  3. 2024Layer- and Timestep-Adaptive Differentiable Token Compression Ratios for Efficient Diffusion Transformerscited by 21
  4. 2024SmoothCache: A Universal Inference Acceleration Technique for Diffusion Transformerscited by 19
  5. 2025CAT Pruning: Cluster-Aware Token Pruning For Text-to-Image Diffusion Modelscited by 16
  6. 2025Model Reveals What to Cache: Profiling-Based Feature Reuse for Video Diffusion Modelscited by 14
  7. 2025A Survey on Cache Methods in Diffusion Models: Toward Efficient Multi-Modal Generationcited by 14
  8. 2025FlexiDiT: Your Diffusion Transformer Can Easily Generate High-Quality Samples with Less Computecited by 12
  9. 2025SkipVAR: Accelerating Visual Autoregressive Modeling via Adaptive Frequency-Aware Skippingcited by 11
  10. 2025ERTACache: Error Rectification and Timesteps Adjustment for Efficient Diffusioncited by 11
  11. 2026SeaCache: Spectral-Evolution-Aware Cache for Accelerating Diffusion Modelscited by 10
  12. 2024Unveiling Redundancy in Diffusion Transformers (DiTs): A Systematic Studycited by 9
  13. 2025RainFusion: Adaptive Video Generation Acceleration via Multi-Dimensional Visual Redundancycited by 7
  14. 2025Foresight: Adaptive Layer Reuse for Accelerated and High-Quality Text-to-Video Generationcited by 6
  15. 2025Forecasting When to Forecast: Accelerating Diffusion Models with Confidence-Gated Taylorcited by 6
  16. 2025Evolutionary Caching to Accelerate Your Off-the-Shelf Diffusion Modelcited by 5
  17. 2024Faster Image2Video Generation: A Closer Look at CLIP Image Embedding's Impact on Spatio-Temporal Cross-Attentionscited by 4
  18. 2024TASR: Timestep-Aware Diffusion Model for Image Super-Resolutioncited by 4
  19. 2025EC-Diff: Fast and High-Quality Edge-Cloud Collaborative Inference for Diffusion Modelscited by 3
  20. 2025Sortblock: Similarity-Aware Feature Reuse for Diffusion Modelcited by 3
  21. 2026ReCache: Learning Budget-Aware Caching Schedules for Diffusion Models via REINFORCEcited by 3
  22. 2024Lazy Layers to Make Fine-Tuned Diffusion Models More Traceablecited by 2
  23. 2025Accelerate High-Quality Diffusion Models with Inner Loop Feedbackcited by 2
  24. 2026ELROND: Exploring and decomposing intrinsic capabilities of diffusion modelscited by 2
  25. 2026Attention, May I Have Your Decision? Localizing Generative Choices in Diffusion Modelscited by 2
  26. 2026Attention Sinks in Diffusion Transformers: A Causal Analysiscited by 2
  27. 2026Budget-Constrained Step-Level Diffusion Cachingcited by 2
  28. 2025Can We Achieve Efficient Diffusion without Self-Attention? Distilling Self-Attention into Convolutionscited by 1
  29. 2025H2-Cache: A Novel Hierarchical Dual-Stage Cache for High-Performance Acceleration of Generative Diffusion Modelscited by 1
  30. 2025RSTR: Reducing SpatioTemporal Redundancy in Diffusion Transformerscited by 1
  31. 2026VDE: Training-Free Accelerating Rectified Flow Model via Velocity Decomposition and Estimationcited by 1
  32. 2026HyperVAttention: Efficient Sparse Attention with Spatio-Temporal Clustering for Video Diffusioncited by 1
  33. 2025ObCLIP: Oblivious CLoud-Device Hybrid Image Generation with Privacy Preservationcited by 0
  34. 2025ETC: training-free diffusion models acceleration with Error-aware Trend Consistencycited by 0
  35. 2025InvarDiff: Cross-Scale Invariance Caching for Accelerated Diffusion Modelscited by 0
  36. 2025CorGi: Contribution-Guided Block-Wise Interval Caching for Training-Free Acceleration of Diffusion Transformerscited by 0
  37. 2026Accelerating Diffusion via Hybrid Data-Pipeline Parallelism Based on Conditional Guidance Schedulingcited by 0
  38. 2026SODA: Sensitivity-Oriented Dynamic Acceleration for Diffusion Transformercited by 0

50 most cited of 54 All 54 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Enhancing Semantic Fidelity in Text-to-Image Synthesis: Attention Regulation in Diffusion Models
  2. 2023Semantic Guidance Tuning for Text-To-Image Diffusion Models
  3. 2023CONFORM: Contrast is All You Need For High-Fidelity Text-to-Image Diffusion Models
  4. 2026FlashBlock: Attention Caching for Efficient Long-Context Block Diffusion
  5. 2024Unveiling and Mitigating Memorization in Text-to-image Diffusion Models through Cross Attention
  6. 2026TPD: Temporal Prior Decoupling for Text-to-Video Diffusion Models
  7. 2023Fast Sampling Through The Reuse Of Attention Maps In Diffusion Models
  8. 2025Visual Autoregressive Models Beat Diffusion Models on Inference Time Scaling