# HEART: Exploiting Head Heterogeneity in Sparse Attention for Video Diffusion arXiv: https://arxiv.org/abs/2605.14513 ยท HTML: https://mlsanity.com/paper/2605.14513/ ## Key facts - Cited by 1 papers in the map (data through 14 Oct 2026). - Placed in Video generation, one of 14 directions in Diffusion models. - Builds on 30 of its 33 references that are in the map. - First posted on arXiv 14 May 2026. ## Placed in - Diffusion models / [Video generation](https://mlsanity.com/diffusion/video-generation/) ## Builds on (most cited first, 30 of 30) - [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018) - [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022) - [Wan: Open and Advanced Large-Scale Video Generative Models](https://mlsanity.com/paper/2503.20314/) (2025) - [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024) - [VBench: Comprehensive Benchmark Suite for Video Generative Models](https://mlsanity.com/paper/2311.17982/) (2023) - [HunyuanVideo: A Systematic Framework For Large Video Generative Models](https://mlsanity.com/paper/2412.03603/) (2024) - [Latte: Latent Diffusion Transformer for Video Generation](https://mlsanity.com/paper/2401.03048/) (2024) - [Sparse VideoGen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsity](https://mlsanity.com/paper/2502.01776/) (2025) - [Step-Video-T2V Technical Report: The Practice, Challenges, and Future of Video Foundation Model](https://mlsanity.com/paper/2502.10248/) (2025) - [Open-Sora 2.0: Training a Commercial-Level Video Generation Model in 200k](https://mlsanity.com/paper/2503.09642/) (2025) - [HunyuanVideo 1.5 Technical Report](https://mlsanity.com/paper/2511.18870/) (2025) - [VisionReward: Fine-Grained Multi-Dimensional Human Preference Learning for Image and Video Generation](https://mlsanity.com/paper/2412.21059/) (2024) - [XAttention: Block Sparse Attention with Antidiagonal Scoring](https://mlsanity.com/paper/2503.16428/) (2025) - [SpargeAttention: Accurate and Training-free Sparse Attention Accelerating Any Model Inference](https://mlsanity.com/paper/2502.18137/) (2025) - [Sparse VideoGen2: Accelerate Video Generation with Sparse Attention via Semantic-Aware Permutation](https://mlsanity.com/paper/2505.18875/) (2025) - [LongCat-Video Technical Report](https://mlsanity.com/paper/2510.22200/) (2025) - [Training-free and Adaptive Sparse Attention for Efficient Long Video Generation](https://mlsanity.com/paper/2502.21079/) (2025) - [Radial Attention: O(nlog n) Sparse Attention with Energy Decay for Long Video Generation](https://mlsanity.com/paper/2506.19852/) (2025) - [VMoBA: Mixture-of-Block Attention for Video Diffusion Models](https://mlsanity.com/paper/2506.23858/) (2025) - [Sparse-vDiT: Unleashing the Power of Sparse Attention to Accelerate Video Diffusion Transformers](https://mlsanity.com/paper/2506.03065/) (2025) - [FreqCa: Accelerating Diffusion Models via Frequency-Aware Caching](https://mlsanity.com/paper/2510.08669/) (2025) - [Bidirectional Sparse Attention for Faster Video Diffusion Training](https://mlsanity.com/paper/2509.01085/) (2025) - [PAROAttention: Pattern-Aware ReOrdering for Efficient Sparse and Quantized Attention in Visual Generation Models](https://mlsanity.com/paper/2506.16054/) (2025) - [PISA: Piecewise Sparse Attention Is Wiser for Efficient Diffusion Transformers](https://mlsanity.com/paper/2602.01077/) (2026) - [Analysis of Attention in Video Diffusion Transformers](https://mlsanity.com/paper/2504.10317/) (2025) - [BLADE: Block-Sparse Attention Meets Step Distillation for Efficient Video Generation](https://mlsanity.com/paper/2508.10774/) (2025) - [LiteAttention: A Temporal Sparse Attention for Diffusion Transformers](https://mlsanity.com/paper/2511.11062/) (2025) - [AdaCluster: Adaptive Query-Key Clustering for Sparse Attention in Video Generation](https://mlsanity.com/paper/2604.18348/) (2026) - [RainFusion2.0: Temporal-Spatial Awareness and Hardware-Efficient Block-wise Sparse Attention](https://mlsanity.com/paper/2512.24086/) (2025) - [Ride the Wave: Precision-Allocated Sparse Attention for Smooth Video Generation](https://mlsanity.com/paper/2604.12219/) (2026) ## Built on by (most cited first, 1 of 1) - [Chorus II: Cross-Request Sparsity Reuse for Efficient Image-to-Video Generation](https://mlsanity.com/paper/2606.25040/) (2026) ## Abstract (arXiv) Sparse attention accelerates video diffusion by allowing each attention head to focus on only a small subset of interactions. Existing methods already construct head-specific sparse patterns conditioned on the input. However, we find that these heads also differ in two less obvious but practically important ways. First, for some heads, sparse attention masks remain stable across many denoising steps, whereas others change rapidly. Second, heads differ substantially in their sensitivity to sparsification: applying the same threshold can induce markedly different errors in the final denoising velocity. Ignoring these differences leads to redundant mask prediction and suboptimal threshold calibration across heads under a global sparsity budget. We present HEART, short for Heterogeneity-Exploiting Adaptive Refresh and Thresholding, a training-free framework that exploits both forms of head heterogeneity. First, Temporal Mask Reuse (TMR) uses a lightweight per-head query-key drift signal to determine whether a cached sparse mask remains reliable across denoising steps, refreshing it only when the drift exceeds a prescribed threshold. Second, Error-guided Budgeted Calibration (EBC) evaluates candidate thresholds offline using a frequency-weighted denoising-velocity error, and assigns each head an appropriate threshold under a global sparsity budget. HEART requires no retraining, weight modification, or sparse-kernel changes, and can be integrated directly into existing sparse-attention pipelines. Across Wan2.1-1.3B, Wan2.1-14B, and HunyuanVideo-13B, HEART consistently pushes the quality--efficiency frontier of advanced sparse attention methods such as XAttention and SVG2 outward.