mlsanityData to 14 Oct 2026

VideoMamba: State Space Model for Efficient Video Understanding

arXiv 2403.06977 ↗First posted 11 Mar 2024cs.CVcited by 328

On the map: State-space sequence models (S4, Mamba) → Video, motion and generative Mamba models

Abstract (arXiv)

Addressing the dual challenges of local redundancy and global dependencies in video understanding, this work innovatively adapts the Mamba to the video domain. The proposed VideoMamba overcomes the limitations of existing 3D convolution neural networks and video transformers. Its linear-complexity operator enables efficient long-term modeling, which is crucial for high-resolution long video understanding. Extensive evaluations reveal VideoMamba's four core abilities: (1) Scalability in the visual domain without extensive dataset pretraining, thanks to a novel self-distillation technique; (2) Sensitivity for recognizing short-term actions even with fine-grained motion differences; (3) Superiority in long-term video understanding, showcasing significant advancements over traditional feature-based models; and (4) Compatibility with other modalities, demonstrating robustness in multi-modal contexts. Through these distinct advantages, VideoMamba sets a new benchmark for video understanding, offering a scalable and efficient solution for comprehensive video understanding. All the code and models are available at https://github.com/OpenGVLab/VideoMamba.

Key facts. Cited by 328 papers in the map (data through 14 Oct 2026). Placed in Video, motion and generative Mamba models, one of 10 directions in State-space sequence models (S4, Mamba). Builds on 76 of its 95 references that are in the map. First posted on arXiv 11 Mar 2024.

Builds on

76 of 95 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Graph-Mamba: Towards Long-Range Graph Sequence Modeling with Selective State Spaces1 Feb

Other references · 75

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  6. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,718
  7. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,703
  8. 2020Training data-efficient image transformers & distillation through attentioncited by 4,586
  9. 2022A ConvNet for the 2020scited by 3,579
  10. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,270
  11. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,465
  12. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
38 more
  1. 2017The Kinetics Human Action Video Datasetcited by 1,909
  2. 2018SlowFast Networks for Video Recognitioncited by 1,867
  3. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,620
  4. 2021ViViT: A Video Vision Transformercited by 1,620
  5. 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,619
  6. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,322
  7. 2021Frozen in Time: A Joint Video and Image Encoder for End-to-End Retrievalcited by 1,177
  8. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,143
  9. 2017The "something something" video database for learning and evaluating visual common sensecited by 1,103
  10. 2014Two-Stream Convolutional Networks for Action Recognition in Videoscited by 1,068
  11. 2024Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Modelcited by 1,056
  12. 2024VMamba: Visual State Space Modelcited by 969
  13. 2019HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clipscited by 950
  14. 2016Temporal Segment Networks: Towards Good Practices for Deep Action Recognitioncited by 902
  15. 2021Multiscale Vision Transformerscited by 897
  16. 2023RWKV: Reinventing RNNs for the Transformer Eracited by 827
  17. 2022Simplified State Space Layers for Sequence Modelingcited by 788
  18. 2019VideoBERT: A Joint Model for Video and Language Representation Learningcited by 703
  19. 2017Localizing Moments in Video with Natural Languagecited by 663
  20. 2021CLIP4Clip: An Empirical Study of CLIP for End to End Video Clip Retrievalcited by 651
  21. 2020X3D: Expanding Architectures for Efficient Video Recognitioncited by 637
  22. 2021VideoCLIP: Contrastive Pre-training for Zero-shot Video-Text Understandingcited by 602
  23. 2017TALL: Temporal Activity Localization via Language Querycited by 584
  24. 2019ActivityNet-QA: A Dataset for Understanding Complex Web Videos via Question Answeringcited by 561
  25. 2022Hungry Hungry Hippos: Towards Language Modeling with State Space Modelscited by 499
  26. 2023InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generationcited by 497
  27. 2017AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actionscited by 449
  28. 2022InternVideo: General Video Foundation Models via Generative and Discriminative Learningcited by 447
  29. 2022Masked Autoencoders As Spatiotemporal Learnerscited by 429
  30. 2024U-Mamba: Enhancing Long-range Dependency for Biomedical Image Segmentationcited by 419
  31. 2019On the Efficacy of Knowledge Distillationcited by 381
  32. 2024MambaIR: A Simple Baseline for Image Restoration with State-Space Modelcited by 352
  33. 2018A Short Note about Kinetics-600cited by 333
  34. 2019COIN: A Large-scale Dataset for Comprehensive Instructional Video Analysiscited by 318
  35. 2019A Short Note on the Kinetics-700 Human Action Datasetcited by 310
  36. 2022Long Range Language Modeling via Gated State Spacescited by 265
  37. 2020FineGym: A Hierarchical Video Dataset for Fine-grained Action Understandingcited by 234
  38. 2021Keeping Your Eye on the Ball: Trajectory Attention in Video Transformerscited by 224

50 most cited of 75 All 75 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 110; 2025: 141; 2026: 77.

328 papers.

Sort:

By area:

  1. 2024VideoTree: Adaptive Tree-based Video Representation for LLM Reasoning on Long Videoscited by 251
  2. 2024Is Mamba Effective for Time Series Forecasting?cited by 114
  3. 2024UniAnimate: Taming Unified Video Diffusion Models for Consistent Human Image Animationcited by 102
  4. 2024ZigMa: A DiT-style Zigzag Mamba Diffusion Modelcited by 84
  5. 2024DiM: Diffusion Mamba for Efficient High-Resolution Image Synthesiscited by 72
  6. 2024Mamba-360: Survey of State Space Models as Transformer Alternative for Long Sequence Modelling: Methods, Applications, and Challengescited by 64
  7. 2024Multi-Scale VMamba: Hierarchy in Hierarchy Visual State Space Modelcited by 62
  8. 2026MEM: Multi-Scale Embodied Memory for Vision Language Action Modelscited by 61
  9. 2024State Space Model for New-Generation Network Alternative to Transformers: A Surveycited by 59
  10. 2024Exploiting Multimodal Spatial-temporal Patterns for Video Object Trackingcited by 57
  11. 2024RainMamba: Enhanced Locality Learning with State Space Models for Video Derainingcited by 56
  12. 2024Vision Mamba: A Comprehensive Survey and Taxonomycited by 54
38 more
  1. 2024Diffusion Models and Representation Learning: A Surveycited by 54
  2. 2024Vision-LSTM: xLSTM as Generic Vision Backbonecited by 51
  3. 2024A Survey on Visual Mambacited by 50
  4. 2025STORM: Token-Efficient Long Video Understanding for Multimodal LLMscited by 49
  5. 2024MambaLLIE: Implicit Retinex-Aware Low Light Enhancement with Global-then-Local State Spacecited by 48
  6. 2024Hamba: Single-view 3D Hand Reconstruction with Graph-guided Bi-Scanning Mambacited by 45
  7. 2024Visual Mamba: A Survey and New Outlookscited by 41
  8. 2024Mamba-R: Vision Mamba ALSO Needs Registerscited by 41
  9. 2024Spatial-Mamba: Effective Visual State Space Models via Structure-aware State Fusioncited by 41
  10. 2024A Survey of Mambacited by 39
  11. 2024SSAMBA: Self-Supervised Audio Representation Learning with Mamba State Space Modelcited by 37
  12. 2025TAPNext: Tracking Any Point (TAP) as Next Token Predictioncited by 37
  13. 2025Vamba: Understanding Hour-Long Videos with Hybrid Mamba-Transformerscited by 36
  14. 2024VSSD: Vision Mamba with Non-Causal State Space Dualitycited by 29
  15. 2024Quamba: A Post-Training Quantization Recipe for Selective State Space Modelscited by 28
  16. 2025Scaling Laws in Patchification: An Image Is Worth 50,176 Tokens And Morecited by 28
  17. 2024SpikeMba: Multi-Modal Spiking Saliency Mamba for Temporal Video Groundingcited by 25
  18. 2024Matten: Video Generation with Mamba-Attentioncited by 24
  19. 2025BIMBA: Selective-Scan Compression for Long-Range Video Question Answeringcited by 24
  20. 2024VFIMamba: Video Frame Interpolation with State Space Modelscited by 22
  21. 2024DGMamba: Domain Generalization via Generalized State Space Modelcited by 21
  22. 2024PointRWKV: Efficient RWKV-Like Model for Hierarchical Point Cloud Learningcited by 21
  23. 2025TREAD: Token Routing for Efficient Architecture-agnostic Diffusion Trainingcited by 21
  24. 2024RhythmMamba: Fast, Lightweight, and Accurate Remote Physiological Measurementcited by 20
  25. 2024SSUMamba: Spatial-Spectral Selective State Space Model for Hyperspectral Image Denoisingcited by 20
  26. 2024VideoLLaMB: Long Streaming Video Understanding with Recurrent Memory Bridgescited by 20
  27. 2026StreamReady: Learning What to Answer and When in Long Streaming Videoscited by 19
  28. 2024Mamba or RWKV: Exploring High-Quality and High-Efficiency Segment Anything Modelcited by 18
  29. 2024MambaTrack: A Simple Baseline for Multiple Object Tracking with State Space Modelcited by 18
  30. 2024Mamba in Vision: A Comprehensive Survey of Techniques and Applicationscited by 18
  31. 2024QuadMamba: Learning Quadtree-based Selective Scan for Visual State Space Modelcited by 18
  32. 2024Sakuga-42M Dataset: Scaling Up Cartoon Researchcited by 17
  33. 2025HierarQ: Task-Aware Hierarchical Q-Former for Enhanced Video Understandingcited by 17
  34. 2025BASKET: A Large-Scale Video Dataset for Fine-Grained Skill Estimationcited by 17
  35. 2024Text-controlled Motion Mamba: Text-Instructed Temporal Grounding of Human Motioncited by 16
  36. 2024A Comprehensive Survey of Mamba Architectures for Medical Image Analysis: Classification, Segmentation, Restoration and Beyondcited by 16
  37. 2025AVS-Mamba: Exploring Temporal and Multi-modal Mamba for Audio-Visual Segmentationcited by 16
  38. 2025Quamba2: A Robust and Scalable Post-training Quantization Framework for Selective State Space Modelscited by 16

50 most cited of 328 All 328 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Video Mamba Suite: State Space Model as a Versatile Alternative for Video Understanding
  2. 2024VideoMamba: Spatio-Temporal Selective State Space Model
  3. 2024MambaOut: Do We Really Need Mamba for Vision?
  4. 2026VideoSEMA: a scalable and efficient Mamba-like attention for video understanding
  5. 2025MoMa: Modulating Mamba for Adapting Image Foundation Models to Video Recognition
  6. 2024LocalMamba: Visual State Space Model with Windowed Selective Scan
  7. 2024Look Every Frame All at Once: Video-Ma2mba for Efficient Long-form Video Understanding with Multi-Axis Gradient Checkpointing
  8. 2026Video Understanding: Through A Temporal Lens