Competition Domains Matter: Selective Head Globalization for Training-Free Sparse Attention in Video Diffusion Transformers
On the map: Diffusion models → Video generation
Abstract (arXiv)
Training-free sparse attention is a practical route to efficient video diffusion transformer inference, yet existing methods typically fix the competition domain at each query row for every attention head. Prior work primarily addresses how to estimate token-level importance within this fixed domain; whether the domain itself should adapt to head heterogeneity remains under-specified. We introduce EG-Sparse, a training-free framework that routes low-entropy heads to a grouped cross-head pool while retaining per-row local selection for high-entropy heads, using only scores already materialized by the sparse pipeline. A controlled diagnostic fixes the ranking score and varies only the competition rule: grouped cross-head selection reduces low-entropy-head reconstruction error by 9.8%, whereas forcing high-entropy heads into an all-head global pool increases it by 7.9%. Across four configurations, EG-Sparse achieves higher video quality while reducing attention density and GPU denoising latency compared with strong training-free sparse-attention baselines. Mechanism ablations further show that selective domain adaptation outperforms uniform, random, and fixed domain-assignment policies. These results support competition-domain adaptation as a distinct design axis within centroid-guided training-free sparse attention.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 26 of its 33 references that are in the map. First posted on arXiv 10 Oct 2026.
Builds on
26 of 33 references.
Closest ancestors · 13
Same direction, newest first
- 2025Sparse VideoGen2: Accelerate Video Generation with Sparse Attention via Semantic-Aware Permutation24 May
- 2025Training-Free Efficient Video Generation via Dynamic Token Carving22 May
- 2025DiTFastAttnV2: Head-wise Attention Compression for Multi-Modality Diffusion Transformers28 Mar
- 2025Training-free and Adaptive Sparse Attention for Efficient Long Video Generation28 Feb
- 2025Fast Video Generation with Sliding Tile Attention6 Feb
- 2025Sparse VideoGen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsity3 Feb
- 2024AsymRnR: Video Diffusion Transformers Acceleration with Asymmetric Reduction and Restoration16 Dec
- 2024Timestep Embedding Tells: It's Time to Cache for Video Diffusion Model28 Nov
- 2024FasterCache: Training-Free Video Diffusion Model Acceleration with High Quality25 Oct
- 2024Pyramidal Flow Matching for Efficient Video Generative Modeling8 Oct
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer12 Aug
- 2024StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Text21 Mar
1 more
Other references · 13
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
- 2024DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Headscited by 288
- 2024SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Accelerationcited by 160
- 2024Real-Time Video Generation with Pyramid Attention Broadcastcited by 149
- 2024Learning-to-Cache: Accelerating Diffusion Transformer via Layer Cachingcited by 142
- 2024SeerAttention: Learning Intrinsic Sparse Attention in Your LLMscited by 124
- 2024SageAttention2: Efficient Attention with Thorough Outlier Smoothing and Per-thread INT4 Quantizationcited by 114
- 2024U-DiTs: Downsample Tokens in U-Shaped Diffusion Transformerscited by 54
- 2025Twilight: Adaptive Attention Sparsity with Hierarchical Top-p Pruningcited by 49
- 2024STIV: Scalable Text and Image Conditioned Video Generationcited by 25
- 2024FlashMask: Efficient and Rich Mask Extension of FlashAttentioncited by 20
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026SPADE: An Input-Adaptive Sparse Attention Engine for Fast Video Diffusion Models Inference
- 2026HEART: Exploiting Head Heterogeneity in Sparse Attention for Video Diffusion
- 2026LoSA: Near-Lossless Sparse Attention for Training-Free Video Diffusion Acceleration
- 2025VSA: Faster Video Diffusion with Trainable Sparse Attention
- 2025Sparse-vDiT: Unleashing the Power of Sparse Attention to Accelerate Video Diffusion Transformers
- 2026LVSA: Training-Free Sparse Attention for Long Video Diffusion
- 2025Bidirectional Sparse Attention for Faster Video Diffusion Training
- 2026SALAD: Achieve High-Sparsity Attention via Efficient Linear Attention Tuning for Video Diffusion Transformer