mlsanityData to 13 Oct 2026

VORTA: Efficient Video Diffusion via Routing Sparse Attention

arXiv 2505.18809 ↗First posted 24 May 2025cs.CVcited by 23

On the map: Diffusion models → Video generation

Abstract (arXiv)

Video diffusion transformers have achieved remarkable progress in high-quality video generation, but remain computationally expensive due to the quadratic complexity of attention over high-dimensional video sequences. Recent acceleration methods enhance the efficiency by exploiting the local sparsity of attention scores; yet they often struggle with accelerating the long-range computation. To address this problem, we propose VORTA, an acceleration framework with two novel components: 1) a sparse attention mechanism that efficiently captures long-range dependencies, and 2) a routing strategy that adaptively replaces full 3D attention with specialized sparse attention variants. VORTA achieves an end-to-end speedup 1.76× without loss of quality on VBench. Furthermore, it can seamlessly integrate with various other acceleration methods, such as model caching and step distillation, reaching up to speedup 14.41× with negligible performance degradation. VORTA demonstrates its efficiency and enhances the practicality of video diffusion transformers in real-world settings. Codes and weights are available at https://github.com/wenhao728/VORTA.

Key facts. Cited by 23 papers in the map (data through 13 Oct 2026). Placed in Video generation, one of 14 directions in Diffusion models. Builds on 55 of its 60 references that are in the map. First posted on arXiv 24 May 2025.

Builds on

55 of 60 references.

Closest ancestors · 8

Same direction, newest first

  1. 2025Training-free and Adaptive Sparse Attention for Efficient Long Video Generation28 Feb
  2. 2025Fast Video Generation with Sliding Tile Attention6 Feb
  3. 2025Sparse VideoGen: Accelerating Video Diffusion Transformers with Spatial-Temporal Sparsity3 Feb
  4. 2024AsymRnR: Video Diffusion Transformers Acceleration with Asymmetric Reduction and Restoration16 Dec
  5. 2024Timestep Embedding Tells: It's Time to Cache for Video Diffusion Model28 Nov
  6. 2024Adaptive Caching for Faster Video Generation with Diffusion Transformers4 Nov
  7. 2024FasterCache: Training-Free Video Diffusion Model Acceleration with High Quality25 Oct
  8. 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer12 Aug

Other references · 47

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,952
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,223
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,777
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,082
  5. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 9,028
  6. 2022Scalable Diffusion Models with Transformerscited by 5,973
  7. 2022Flow Matching for Generative Modelingcited by 5,542
  8. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,130
  9. 2018Neural Ordinary Differential Equationscited by 3,750
  10. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,637
  11. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,607
  12. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,463
35 more
  1. 2020Longformer: The Long-Document Transformercited by 3,393
  2. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,610
  3. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,570
  4. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,056
  5. 2023Consistency Modelscited by 1,797
  6. 2023VBench: Comprehensive Benchmark Suite for Video Generative Modelscited by 1,775
  7. 2020Big Bird: Transformers for Longer Sequencescited by 1,681
  8. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,625
  9. 2022SmoothQuant: Accurate and Efficient Post-Training Quantization for Large Language Modelscited by 1,365
  10. 2023One-step Diffusion with Distribution Matching Distillationcited by 991
  11. 2022DPM-Solver++: Fast Solver for Guided Sampling of Diffusion Probabilistic Modelscited by 931
  12. 2023Vision Transformers Need Registerscited by 850
  13. 2023Adversarial Diffusion Distillationcited by 751
  14. 2022On Distillation of Guided Diffusion Modelscited by 721
  15. 2024Movie Gen: A Cast of Media Foundation Modelscited by 624
  16. 2023UniPC: A Unified Predictor-Corrector Framework for Fast Sampling of Diffusion Modelscited by 444
  17. 2023InstaFlow: One Step is Enough for High-Quality Diffusion-Based Text-to-Image Generationcited by 391
  18. 2024MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attentioncited by 378
  19. 2023FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attentioncited by 313
  20. 2024Open-Sora Plan: Open-Source Large Video Generation Modelcited by 288
  21. 2024DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Headscited by 284
  22. 2023Token Merging for Fast Stable Diffusioncited by 223
  23. 2017BlockDrop: Dynamic Inference Paths in Residual Networkscited by 187
  24. 2015Conditional Computation in Neural Networks for faster modelscited by 167
  25. 2017Convolutional Networks with Adaptive Inference Graphscited by 147
  26. 2024Real-Time Video Generation with Pyramid Attention Broadcastcited by 144
  27. 2024Phased Consistency Modelscited by 85
  28. 2020An Overview of Neural Network Compressioncited by 59
  29. 2017Deciding How to Decide: Dynamic Routing in Artificial Neural Networkscited by 33
  30. 2023SE(3) Diffusion Model-based Point Cloud Registration for Robust 6D Object Pose Estimationcited by 32
  31. 2024PersonalVideo: High ID-Fidelity Video Customization without Dynamic and Semantic Degradationcited by 29
  32. 2024Selective Attention Improves Transformercited by 26
  33. 2025MagicID: Hybrid Preference Optimization for ID-Consistent and Dynamic-Preserved Video Customizationcited by 17
  34. 2023Local Conditional Controlling for Text-to-Image Diffusion Modelscited by 13
  35. 2025SUDO: Enhancing Text-to-Image Diffusion Models with Self-Supervised Direct Preference Optimizationcited by 3

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2025: 5; 2026: 18.

23 papers.

Sort:

By area:

  1. 2025End-to-End Training for Autoregressive Video Diffusion via Self-Resamplingcited by 46
  2. 2026SLA2: Sparse-Linear Attention with Learnable Routing and QATcited by 22
  3. 2025Bidirectional Sparse Attention for Faster Video Diffusion Trainingcited by 19
  4. 2026SpargeAttention2: Trainable Sparse Attention via Hybrid Top-k+Top-p Masking and Distillation Fine-Tuningcited by 18
  5. 2026PISA: Piecewise Sparse Attention Is Wiser for Efficient Diffusion Transformerscited by 17
  6. 2026Efficient Autoregressive Video Diffusion with Dummy Headcited by 16
  7. 2025LinVideo: A Post-Training Framework towards O(n) Attention in Efficient Video Generationcited by 10
  8. 2026SVG-EAR: Parameter-Free Linear Compensation for Sparse Video Generation via Error-aware Routingcited by 10
  9. 2025RegionE: Adaptive Region-Aware Generation for Efficient Image Editingcited by 6
  10. 2026Efficient Video Diffusion Models: Advancements and Challengescited by 6
  11. 2026Video Generation Models as World Models: Efficient Paradigms, Architectures and Algorithmscited by 5
  12. 2026VMonarch: Efficient Video Diffusion Transformers with Structured Attentioncited by 3
11 more
  1. 2026SPA-Cache: Singular Proxies for Adaptive Caching in Diffusion Language Modelscited by 1
  2. 2026DynamicRad: Content-Adaptive Sparse Attention for Long Video Diffusioncited by 1
  3. 2026Veda: Scalable Video Diffusion via Distilled Sparse Attentioncited by 1
  4. 2026Chorus II: Cross-Request Sparsity Reuse for Efficient Image-to-Video Generationcited by 1
  5. 2026HyperVAttention: Efficient Sparse Attention with Spatio-Temporal Clustering for Video Diffusioncited by 1
  6. 2026Partition the Support, Reconstruct the Residual: Training-Free Sparse Attention for Video Generation and World Modelscited by 1
  7. 2025Lightning Fast Caching-based Parallel Denoising Prediction for Accelerating Talking Head Generationcited by 0
  8. 2026Near-Oracle KV Selection via Pre-hoc Sparsity for Long-Context Inferencecited by 0
  9. 2026Training-free sparse attention based on cumulative energy filteringcited by 0
  10. 2026SCOPE: Subspace Clustering with Online Per-Head Top-K Estimation for Sparse Video Attentioncited by 0
  11. 2026Parameterized Stripe Attention for Efficient Video Generationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Sparse-vDiT: Unleashing the Power of Sparse Attention to Accelerate Video Diffusion Transformers
  2. 2025Efficient-vDiT: Efficient Video Diffusion Transformers With Attention Tile
  3. 2026SPADE: An Input-Adaptive Sparse Attention Engine for Fast Video Diffusion Models Inference
  4. 2025Attention Surgery: An Efficient Recipe to Linearize Your Video Diffusion Transformer
  5. 2026SALAD: Achieve High-Sparsity Attention via Efficient Linear Attention Tuning for Video Diffusion Transformer
  6. 2025VSA: Faster Video Diffusion with Trainable Sparse Attention
  7. 2025DraftAttention: Fast Video Diffusion via Low-Resolution Attention Guidance
  8. 2026LVSA: Training-Free Sparse Attention for Long Video Diffusion