SANA-Video 2.0: Hybrid Linear Attention with Attention Residuals for Efficient Video Generation
On the map: Diffusion models → Video generation
Abstract (arXiv)
We introduce SANA-Video 2.0, a hybrid video diffusion transformer instantiated at 5B and 14B scales under a unified architecture. Designed to generate high-quality video up to 720p on a single GPU, SANA-Video 2.0 matches full-softmax video DiTs in quality while retaining the favorable long-sequence scaling of linear attention. To avoid quadratic attention throughout, Hybrid Linear-Softmax Attention combines gated linear attention for O(N)-dominated mixing with periodic gated-softmax anchors at a 3:1 ratio, restoring the full-rank token interactions that pure linear attention lacks. To propagate these refreshed representations across depth, Block Attention Residuals (AttnRes) route completed block summaries into later linear layers, enabling anchor-feature reuse and boosting deep-layer effective rank by ~12%. Through from-scratch training, SANA-Video 2.0 learns the complete hybrid directly rather than linearizing pretrained models, with reduced-resolution proxy studies establishing 25% softmax as the optimal quality-efficiency trade-off. With 40-step sampling, SANA-Video 2.0 achieves a VBench score of 84.30 in 13.2s at 480p on a single H100, remaining competitive with far larger softmax video DiTs at a fraction of the latency. Its compiled DiT forward pass is 3.2x faster than a matched full-softmax baseline at 720p/60s, a gap that expands with video duration. Furthermore, full-stack Sol-Engine optimization (kernel fusion, caching, and sparse attention) accelerates this hardware-friendly backbone by a further 3.58x, bringing the 5B pipeline to 13.06s at 720p/5s and making it 120x faster than Wan 2.2-A14B on one H100. Overall, our hybrid design recovers softmax-level expressiveness at substantially reduced cost, unlocking scalable long, high resolution video generation.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 47 of its 51 references that are in the map. First posted on arXiv 23 Jul 2026.
Builds on
47 of 51 references.
Closest ancestors · 11
Same direction, newest first
- 2026Sol Video Inference Engine: Agent-Native Full-Stack Acceleration Framework for Efficient Video Generation21 Jun
- 2026SANA-Streaming: Real-time Streaming Video Editing with Hybrid Diffusion Transformer28 May
- 2026Bernini: Latent Semantic Planning for Video Diffusion21 May
- 2026PISA: Piecewise Sparse Attention Is Wiser for Efficient Diffusion Transformers1 Feb
- 2025Attention Surgery: An Efficient Recipe to Linearize Your Video Diffusion Transformer29 Sep
- 2025SANA-Video: Efficient Video Generation with Block Linear Diffusion Transformer29 Sep
- 2025Radial Attention: O(nlog n) Sparse Attention with Energy Decay for Long Video Generation24 Jun
- 2025Sparse VideoGen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsity3 Feb
- 2024LTX-Video: Realtime Video Latent Diffusion30 Dec
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer12 Aug
- 2024DiG: Scalable and Efficient Diffusion Models with Gated Linear Attention28 May
Other references · 36
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2022Flow Matching for Generative Modelingcited by 5,702
- 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
- 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
- 2023Sigmoid Loss for Language Image Pre-Trainingcited by 3,109
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
- 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 2,022
- 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
- 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,822
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,660
- 2024VMamba: Visual State Space Modelcited by 969
- 2023Diffusion Model Alignment Using Direct Preference Optimizationcited by 790
24 more
- 2023Gated Linear Attention Transformers with Hardware-Efficient Trainingcited by 514
- 2025Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attentioncited by 407
- 2024VideoMamba: State Space Model for Efficient Video Understandingcited by 328
- 2022Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectivescited by 320
- 2025Gated Attention for Large Language Models: Non-linearity, Sparsity, and Attention-Sink-Freecited by 283
- 2025MAGI-1: Autoregressive Video Generation at Scalecited by 276
- 2022Unifying Flow, Stereo and Depth Estimationcited by 275
- 2024Simple linear attention language models balance the recall-throughput tradeoffcited by 208
- 2025Kimi Linear: An Expressive, Efficient Attention Architecturecited by 196
- 2020TransNet V2: An effective deep network architecture for fast shot transition detectioncited by 171
- 2026Cosmos 3: Omnimodal World Models for Physical AIcited by 165
- 2025Open-Sora 2.0: Training a Commercial-Level Video Generation Model in 200kcited by 163
- 2026Seedance 2.0: Advancing Video Generation for World Complexitycited by 160
- 2025Teaching Large Language Models to Regress Accurate Image Quality Scores using Score Distributioncited by 141
- 2025SpargeAttention: Accurate and Training-free Sparse Attention Accelerating Any Model Inferencecited by 120
- 2024DiM: Diffusion Mamba for Efficient High-Resolution Image Synthesiscited by 72
- 2026Attention Residualscited by 65
- 2026SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformercited by 55
- 2024Breaking the Low-Rank Dilemma of Linear Attentioncited by 31
- 2026Self-Supervised Flow Matching for Scalable Multi-Modal Synthesiscited by 31
- 2025UniPercept: Towards Unified Perceptual-Level Image Understanding across Aesthetics, Quality, Structure, and Texturecited by 22
- 2026Scaling Mixture-of-Experts Video Pretraining for Embodied Intelligencecited by 15
- 2026HPSv3++: Scaling Reward Models Across the Full Spectrum of Diffusion Model Capabilitiescited by 5
- 2026Beyond the Golden Data: Resolving the Motion-Vision Quality Dilemma via Timestep Selective Trainingcited by 3
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026LVSA: Training-Free Sparse Attention for Long Video Diffusion
- 2026ReHyAt: Recurrent Hybrid Attention for Video Diffusion Transformers
- 2025VSA: Faster Video Diffusion with Trainable Sparse Attention
- 2025SLA: Beyond Sparsity in Diffusion Transformers via Fine-Tunable Sparse-Linear Attention
- 2026LoSA: Near-Lossless Sparse Attention for Training-Free Video Diffusion Acceleration
- 2025Bidirectional Sparse Attention for Faster Video Diffusion Training
- 2026SALAD: Achieve High-Sparsity Attention via Efficient Linear Attention Tuning for Video Diffusion Transformer
- 2025DraftAttention: Fast Video Diffusion via Low-Resolution Attention Guidance