# BIMBA: Selective-Scan Compression for Long-Range Video Question Answering

arXiv: https://arxiv.org/abs/2503.09590 · HTML: https://mlsanity.com/paper/2503.09590/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 71 of its 86 references that are in the map.
- First posted on arXiv 12 Mar 2025.

## Builds on (most cited first, 30 of 71)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [Sigmoid Loss for Language Image Pre-Training](https://mlsanity.com/paper/2303.15343/) (2023)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models](https://mlsanity.com/paper/2304.10592/) (2023)
- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://mlsanity.com/paper/2312.14238/) (2023)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [Ego4D: Around the World in 3,000 Hours of Egocentric Video](https://mlsanity.com/paper/2110.07058/) (2021)
- [Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understanding](https://mlsanity.com/paper/2306.02858/) (2023)
- [Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis](https://mlsanity.com/paper/2405.21075/) (2024)
- [Video-LLaVA: Learning United Visual Representation by Alignment Before Projection](https://mlsanity.com/paper/2311.10122/) (2023)
- [MVBench: A Comprehensive Multi-modal Video Understanding Benchmark](https://mlsanity.com/paper/2311.17005/) (2023)
- [Perceiver: General Perception with Iterative Attention](https://mlsanity.com/paper/2103.03206/) (2021)
- [Video-ChatGPT: Towards Detailed Video Understanding via Large Vision and Language Models](https://mlsanity.com/paper/2306.05424/) (2023)
- [Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model](https://mlsanity.com/paper/2401.09417/) (2024)
- [mPLUG-Owl: Modularization Empowers Large Language Models with Multimodality](https://mlsanity.com/paper/2304.14178/) (2023)
- [VideoChat: Chat-Centric Video Understanding](https://mlsanity.com/paper/2305.06355/) (2023)
- [VideoLLaMA 2: Advancing Spatial-Temporal Modeling and Audio Understanding in Video-LLMs](https://mlsanity.com/paper/2406.07476/) (2024)

## Built on by (most cited first, 24 of 24)

- [ReAgent-V: A Reward-Driven Multi-Agent Framework for Video Understanding](https://mlsanity.com/paper/2506.01300/) (2025)
- [Vamba: Understanding Hour-Long Videos with Hybrid Mamba-Transformers](https://mlsanity.com/paper/2503.11579/) (2025)
- [Video-RTS: Rethinking Reinforcement Learning and Test-Time Scaling for Efficient and Enhanced Video Reasoning](https://mlsanity.com/paper/2507.06485/) (2025)
- [VideoEval-Pro: Robust and Realistic Long Video Understanding Evaluation](https://mlsanity.com/paper/2505.14640/) (2025)
- [SiLVR: A Simple Language-based Video Reasoning Framework](https://mlsanity.com/paper/2505.24869/) (2025)
- [AVATAR: Reinforcement Learning to See, Hear, and Reason Over Video](https://mlsanity.com/paper/2508.03100/) (2025)
- [ReMoRa: Multimodal Large Language Model based on Refined Motion Representation for Long-Video Understanding](https://mlsanity.com/paper/2602.16412/) (2026)
- [Small Vision-Language Models are Smart Compressors for Long Video Understanding](https://mlsanity.com/paper/2604.08120/) (2026)
- [TimeViper: A Hybrid Mamba-Transformer Vision-Language Model for Efficient Long Video Understanding](https://mlsanity.com/paper/2511.16595/) (2025)
- [Personal Visual Context Learning in Large Multimodal Models](https://mlsanity.com/paper/2605.10936/) (2026)
- [Do Modern Video-LLMs Need to Listen? A Benchmark Audit and Scalable Remedy](https://mlsanity.com/paper/2509.17901/) (2025)
- [Image-to-Video Transfer Learning based on Image-Language Foundation Models: A Comprehensive Survey](https://mlsanity.com/paper/2510.10671/) (2025)
- [RT-NeuS: Towards Real-Time Neuro-Symbolic Video Understanding via Adaptive Temporal Verification](https://mlsanity.com/paper/2602.23553/) (2026)
- [AdaptToken: Entropy-based Adaptive Token Selection for MLLM Long Video Understanding](https://mlsanity.com/paper/2603.28696/) (2026)
- [MemoryCard: Topic-Aware Multi-Modal Clue Compression for Long-Video Question Answering](https://mlsanity.com/paper/2606.05917/) (2026)
- [Incentivizing Vision Language Models to Search for Long Video Question Answering](https://mlsanity.com/paper/2607.02959/) (2026)
- [HieraMamba: Video Temporal Grounding via Hierarchical Anchor-Mamba Pooling](https://mlsanity.com/paper/2510.23043/) (2025)
- [HiCrew: Hierarchical Reasoning for Long-Form Video Understanding via Question-Aware Multi-Agent Collaboration](https://mlsanity.com/paper/2604.21444/) (2026)
- [SVI-Bench: A Dynamic Microworld for Strategic Video Intelligence](https://mlsanity.com/paper/2605.31529/) (2026)
- [ReVA: A Scene-Centric Dataset Beyond Repetition for Remote Sensing Video Question Answering](https://mlsanity.com/paper/2609.35507/) (2026)
- [Searching Videos as Trees: Self-Correcting Agents for Grounded Long Video QA](https://mlsanity.com/paper/2607.16189/) (2026)
- [LongVU-TTT: Causal Test-Time Training for Visual Resampling in Long Video Understanding](https://mlsanity.com/paper/2608.25729/) (2026)
- [dKFD: Phase-Structured Evidence Allocation for Fixed-Budget Localized Event Understanding](https://mlsanity.com/paper/2609.33083/) (2026)
- [WIRE: Weight Space Implicit Representation Encoding of Frozen Video LLMs](https://mlsanity.com/paper/2610.17355/) (2026)

## Abstract (arXiv)

Video Question Answering (VQA) in long videos poses the key challenge of extracting relevant information and modeling long-range dependencies from many redundant frames. The self-attention mechanism provides a general solution for sequence modeling, but it has a prohibitive cost when applied to a massive number of spatiotemporal tokens in long videos. Most prior methods rely on compression strategies to lower the computational cost, such as reducing the input length via sparse frame sampling or compressing the output sequence passed to the large language model (LLM) via space-time pooling. However, these naive approaches over-represent redundant information and often miss salient events or fast-occurring space-time patterns. In this work, we introduce BIMBA, an efficient state-space model to handle long-form videos. Our model leverages the selective scan algorithm to learn to effectively select critical information from high-dimensional video and transform it into a reduced token sequence for efficient LLM processing. Extensive experiments demonstrate that BIMBA achieves state-of-the-art accuracy on multiple long-form VQA benchmarks, including PerceptionTest, NExT-QA, EgoSchema, VNBench, LongVideoBench, and Video-MME. Code, and models are publicly available at https://sites.google.com/view/bimba-mllm.