mlsanityData to 7 Oct 2026

VideoNSA: Native Sparse Attention Scales Video Understanding

arXiv 2510.02295 ↗First posted 2 Oct 2025cs.CV cs.AI cs.LGcited by 10

On the map: Multimodal LLMs / vision-language models → Video understanding with MLLMs

Abstract (arXiv)

Video understanding in multimodal language models remains limited by context length: models often miss key transition frames and struggle to maintain coherence across long time scales. To address this, we adapt Native Sparse Attention (NSA) to video-language models. Our method, VideoNSA, adapts Qwen2.5-VL through end-to-end training on a 216K video instruction dataset. We employ a hardware-aware hybrid approach to attention, preserving dense attention for text, while employing NSA for video. Compared to token-compression and training-free sparse baselines, VideoNSA achieves improved performance on long-video understanding, temporal reasoning, and spatial benchmarks. Further ablation analysis reveals four key findings: (1) reliable scaling to 128K tokens; (2) an optimal global-local attention allocation at a fixed budget; (3) task-dependent branch usage patterns; and (4) the learnable combined sparse attention help induce dynamic attention sinks. Project Page: https://enxinsong.com/VideoNSA-web/, Code: https://github.com/Espere-1119-Song/VideoNSA

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 80 of its 108 references that are in the map. First posted on arXiv 2 Oct 2025.

Builds on

80 of 108 references.

Closest ancestors · 10

Same direction, newest first

  1. 2025StreamMem: Query-Agnostic KV Cache Memory for Streaming Video Understanding21 Aug
  2. 2025Video-XL-2: Towards Very Long-Video Understanding Through Task-Aware KV Sparsification24 Jun
  3. 2025VideoEval-Pro: Robust and Realistic Long Video Understanding Evaluation20 May
  4. 2025Does Your Vision-Language Model Get Lost in the Long Video Sampling Dilemma?16 Mar
  5. 2025InternVideo2.5: Empowering Video MLLMs with Long and Rich Context Modeling21 Jan
  6. 2024VideoChat-Flash: Hierarchical Compression for Long-Context Video Modeling31 Dec
  7. 2024Video-XL: Extra-Long Vision Language Model for Hour-Scale Video Understanding22 Sep
  8. 2024LongVILA: Scaling Long-Context Visual Language Models for Long Videos19 Aug
  9. 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding22 Jul
  10. 2024Long Context Transfer from Language to Vision24 Jun

Other references · 70

  1. 2025Qwen2.5-VL Technical Reportcited by 5,418
  2. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,581
  3. 2020Longformer: The Long-Document Transformercited by 3,362
  4. 2024LLaVA-OneVision: Easy Visual Task Transfercited by 2,669
  5. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  6. 2023Efficient Streaming Language Models with Attention Sinkscited by 2,115
  7. 2024Expanding Performance Boundaries of Open-Source Multimodal Models with Model, Data, and Test-Time Scalingcited by 1,573
  8. 2023GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpointscited by 1,311
  9. 2021Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolationcited by 1,085
  10. 2023Extending Context Window of Large Language Models via Positional Interpolationcited by 721
  11. 2024An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Modelscited by 629
  12. 2024Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spacescited by 580
38 more
  1. 2023MovieChat: From Dense Token to Sparse Memory for Long Video Understandingcited by 536
  2. 2025VideoLLaMA 3: Frontier Multimodal Foundation Models for Image and Video Understandingcited by 502
  3. 2024VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Modelscited by 443
  4. 2025Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attentioncited by 391
  5. 2025Kimi-VL Technical Reportcited by 381
  6. 2024MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attentioncited by 375
  7. 2023Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understandingcited by 368
  8. 2024SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuningcited by 345
  9. 2024LMMs-Eval: Reality Check on the Evaluation of Large Multimodal Modelscited by 344
  10. 2024VisionZip: Longer is Better but Not Necessary in Vision Language Modelscited by 298
  11. 2024When Attention Sink Emerges in Language Models: An Empirical Viewcited by 200
  12. 2024AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmarkcited by 137
  13. 2025XAttention: Block Sparse Attention with Antidiagonal Scoringcited by 131
  14. 2024SeerAttention: Learning Intrinsic Sparse Attention in Your LLMscited by 121
  15. 2025FlexPrefill: A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inferencecited by 121
  16. 2025On the Generalization of SFT: A Reinforcement Learning Perspective with Reward Rectificationcited by 121
  17. 2025VSA: Faster Video Diffusion with Trainable Sparse Attentioncited by 100
  18. 2024TemporalBench: Benchmarking Fine-grained Temporal Understanding for Multimodal Video Modelscited by 96
  19. 2025Token Pruning in Multimodal Large Language Models: Are We Solving the Right Problem?cited by 76
  20. 2024MovieChat+: Question-aware Sparse Memory for Long Video Question Answeringcited by 73
  21. 2024TOMATO: Assessing Visual Temporal Reasoning Capabilities in Multimodal Foundation Modelscited by 71
  22. 2024Number it: Temporal Grounding Videos like Flipping Mangacited by 65
  23. 2023Exploring Diverse In-Context Configurations for Image Captioningcited by 63
  24. 2024SCBench: A KV Cache-Centric Analysis of Long-Context Methodscited by 59
  25. 2025SLA: Beyond Sparsity in Diffusion Transformers via Fine-Tunable Sparse-Linear Attentioncited by 55
  26. 2025VScan: Rethinking Visual Token Reduction for Efficient Large Vision-Language Modelscited by 42
  27. 2025RocketKV: Accelerating Long-Context LLM Inference via Two-Stage KV Cache Compressioncited by 41
  28. 2025Video-MMLU: A Massive Multi-Discipline Lecture Understanding Benchmarkcited by 37
  29. 2024ReTaKe: Reducing Temporal and Knowledge Redundancy for Long Video Understandingcited by 33
  30. 2025InfLLM-V2: Dense-Sparse Switchable Attention for Seamless Short-to-Long Adaptationcited by 31
  31. 2025Multi-Granular Spatio-Temporal Token Merging for Training-Free Acceleration of Video LLMscited by 30
  32. 2025Transformers are Graph Neural Networkscited by 29
  33. 2025MMTok: Multimodal Coverage Maximization for Efficient Inference of VLMscited by 29
  34. 2025MMInference: Accelerating Pre-filling for Long-Context VLMs via Modality-Aware Permutation Sparse Attentioncited by 26
  35. 2024PVC: Progressive Visual Token Compression for Unified Image and Video Processing in Large Vision-Language Modelscited by 25
  36. 2025Reinforcement Learning Tuning for VideoLLMs: Reward Design and Data Efficiencycited by 25
  37. 2025A Survey of Token Compression for Efficient Multimodal Large Language Modelscited by 22
  38. 2025VFlowOpt: A Token Pruning Framework for LMMs with Visual Information Flow-Guided Optimizationcited by 21

50 most cited of 70 All 70 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 7.

10 papers.

  1. 2025InfiniteVL: Synergizing Linear and Sparse Attention for Highly-Efficient, Unlimited-Input Vision-Language Modelscited by 9
  2. 2025Video-LMM Post-Training: A Deep Dive into Video Reasoning with Large Multimodal Modelscited by 8
  3. 2026Efficient Inference for Large Vision-Language Models: Bottlenecks, Techniques, and Prospectscited by 8
  4. 2026Video Generation Models as World Models: Efficient Paradigms, Architectures and Algorithmscited by 5
  5. 2026Watch, Remember, Reason: Human-View Video Understanding with MLLMscited by 3
  6. 2026MuseBench: Benchmarking Intent-Level Audiovisual Arts Understanding in MLLMscited by 1
  7. 2025Towards Effective Long Video Understanding of Multimodal Large Language Models via One-shot Clip Retrievalcited by 0
  8. 2026A Vision for Multisensory Intelligence: Sensing, Science, and Synergycited by 0
  9. 2026Learning to See What You Need: Gaze Attention for Multimodal Large Language Modelscited by 0
  10. 2026OmniTide: Co-Designing Algorithms and Systems for Efficient On-Device Omni-LLM Streamingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Optimizing Native Sparse Attention with Latent Attention and Local Global Alternating Strategies
  2. 2026Temporal Gains, Spatial Costs: Revisiting Video Fine-Tuning in Multimodal Large Language Models
  3. 2026LVSA: Training-Free Sparse Attention for Long Video Diffusion
  4. 2025CS-VLM: Compressed Sensing Attention for Efficient Vision-Language Representation Learning
  5. 2025FiLA-Video: Spatio-Temporal Compression for Fine-Grained Long Video Understanding
  6. 2026Sink-Token-Aware Pruning for Fine-Grained Video Understanding in Efficient Video LLMs
  7. 2024Fewer Tokens and Fewer Videos: Extending Video Understanding Abilities in Large Vision-Language Models
  8. 2025AdaTP: Attention-Debiased Token Pruning for Video Large Language Models