mlsanityData to 14 Oct 2026

APB-V: Accelerating Long-Video Understanding via Sequence-Parallelism-aware Approximate Attention

arXiv 2601.21444 ↗First posted 29 Jan 2026cs.CV cs.AI cs.CLcited by 0

On the map: Multimodal LLMs / vision-language models → Video understanding with MLLMs

Abstract (arXiv)

The efficiency of long-video inference remains a critical bottleneck, mainly due to the dense computation in the prefill stage of Large Multimodal Models (LMMs). Existing methods either compress visual embeddings or apply sparse attention on a single GPU, yielding limited acceleration or degraded performance and restricting LMMs from handling longer, more complex videos. To overcome these issues, we propose APB-V, a sequence-parallel framework with optimized attention that accelerates long-video inference across multiple GPUs. By distributing approximate attention, APB-V reduces computation and increases parallelism, enabling efficient processing of more visual embeddings without compression and thereby improving task performance. System-level optimizations, such as load balancing and fused forward passes, further unleash the potential of APB-V, delivering speedups of 12.72x, 1.70x, and 1.18x over FlashAttn, ZigZagRing, and APB, without notable performance loss. Code available at https://github.com/thunlp/APB

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Video understanding with MLLMs, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 37 of its 44 references that are in the map. First posted on arXiv 29 Jan 2026.

Builds on

37 of 44 references.

Closest ancestors · 4

Same direction, newest first

  1. 2024LongVU: Spatiotemporal Adaptive Compression for Long Video-Language Understanding22 Oct
  2. 2024LongVILA: Scaling Long-Context Visual Language Models for Long Videos19 Aug
  3. 2024LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding22 Jul
  4. 2024Needle In A Video Haystack: A Scalable Synthetic Evaluator for Video MLLMs13 Jun

Other references · 33

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2025Qwen2.5-VL Technical Reportcited by 5,587
  4. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,598
  5. 2024RULER: What's the Real Context Size of Your Long-Context Language Models?cited by 1,346
  6. 2024SnapKV: LLM Knows What You are Looking for Before Generationcited by 880
  7. 2021NExT-QA:Next Phase of Question-Answering to Explaining Temporal Actionscited by 744
  8. 2024An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Modelscited by 642
  9. 2024KIVI: A Tuning-Free Asymmetric 2bit Quantization for KV Cachecited by 637
  10. 2024MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attentioncited by 383
  11. 2024DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Headscited by 288
  12. 2024InfiniGen: Efficient Generative Inference of Large Language Models with Dynamic KV Cache Managementcited by 245
21 more
  1. 2023ProPainter: Improving Propagation and Transformer for Video Inpaintingcited by 197
  2. 2024InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memorycited by 197
  3. 2024SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Accelerationcited by 160
  4. 2025XAttention: Block Sparse Attention with Antidiagonal Scoringcited by 137
  5. 2024ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inferencecited by 134
  6. 2024SlowFast-LLaVA: A Strong Training-Free Baseline for Video Large Language Modelscited by 125
  7. 2024SageAttention2: Efficient Attention with Thorough Outlier Smoothing and Per-thread INT4 Quantizationcited by 114
  8. 2025TimeChat-Online: 80% Visual Tokens are Naturally Redundant in Streaming Videoscited by 97
  9. 2025MiniCPM4: Ultra-Efficient LLMs on End Devicescited by 52
  10. 2024Don't Look Twice: Faster Video Transformers with Run-Length Tokenizationcited by 43
  11. 2024Dynamic-VLM: Simple Dynamic Visual Token Compression for VideoLLMcited by 32
  12. 2025SlowFast-LLaVA-1.5: A Family of Token-Efficient Video Large Language Models for Long-Form Video Understandingcited by 30
  13. 2024Video Super-Resolution Transformer with Masked Inter&Intra-Frame Attentioncited by 29
  14. 2024InfiniPot: Infinite Context Processing on Memory-Constrained LLMscited by 27
  15. 2025MMInference: Accelerating Pre-filling for Long-Context VLMs via Modality-Aware Permutation Sparse Attentioncited by 26
  16. 2024Star Attention: Efficient LLM Inference over Long Sequencescited by 25
  17. 2024Video Token Merging for Long-form Video Understandingcited by 23
  18. 2024Principles of Visual Tokens for Efficient Video Understandingcited by 7
  19. 2024Efficient Long Video Tokenization via Coordinate-based Patch Reconstructioncited by 6
  20. 2024SkipViT: Speeding Up Vision Transformers with a Token-Level Skip Connectioncited by 5
  21. 2025APB: Accelerating Distributed Long-Context Inference by Passing Compressed Context Blocks across GPUscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Vamba: Understanding Hour-Long Videos with Hybrid Mamba-Transformers
  2. 2024Look Every Frame All at Once: Video-Ma2mba for Efficient Long-form Video Understanding with Multi-Axis Gradient Checkpointing
  3. 2025LV-XAttn: Distributed Cross-Attention for Long Visual Inputs in Multimodal Large Language Models
  4. 2025APVR: Hour-Level Long Video Understanding with Adaptive Pivot Visual Information Retrieval
  5. 2026MegaSlide-DiT: Memory-Centric Adaptation and Deformable Local Attention for Efficient Video Diffusion
  6. 2025PEVLM: Parallel Encoding for Vision-Language Models
  7. 2025db-SP: Accelerating Sparse Attention for Visual Generative Models with Dual-Balanced Sequence Parallelism
  8. 2025Enabling Disaggregated Multi-Stage MLLM Inference via GPU-Internal Scheduling and Resource Sharing