mlsanityData to 7 Oct 2026

VideoMamba: Spatio-Temporal Selective State Space Model

arXiv 2407.08476 ↗First posted 11 Jul 2024cs.CVcited by 25

On the map: State-space sequence models (S4, Mamba) → Video, motion and generative Mamba models

Abstract (arXiv)

We introduce VideoMamba, a novel adaptation of the pure Mamba architecture, specifically designed for video recognition. Unlike transformers that rely on self-attention mechanisms leading to high computational costs by quadratic complexity, VideoMamba leverages Mamba's linear complexity and selective SSM mechanism for more efficient processing. The proposed Spatio-Temporal Forward and Backward SSM allows the model to effectively capture the complex relationship between non-sequential spatial and sequential temporal information in video. Consequently, VideoMamba is not only resource-efficient but also effective in capturing long-range dependency in videos, demonstrated by competitive performance and outstanding efficiency on a variety of video understanding benchmarks. Our work highlights the potential of VideoMamba as a powerful tool for video understanding, offering a simple yet effective baseline for future research in video analysis.

Key facts. Cited by 25 papers in the map (data through 7 Oct 2026). Placed in Video, motion and generative Mamba models, one of 10 directions in State-space sequence models (S4, Mamba). Builds on 48 of its 54 references that are in the map. First posted on arXiv 11 Jul 2024.

Builds on

48 of 54 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Selective Structured State-Spaces for Long-Form Video Understanding25 Mar

Other references · 47

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2020End-to-End Object Detection with Transformerscited by 7,384
  4. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  5. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  6. 2017Quo Vadis, Action Recognition? A New Model and the Kinetics Datasetcited by 3,259
  7. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
  8. 2017Non-local Neural Networkscited by 2,290
  9. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  10. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  11. 2017The Kinetics Human Action Video Datasetcited by 1,894
  12. 2018SlowFast Networks for Video Recognitioncited by 1,862
35 more
  1. 2021ViViT: A Video Vision Transformercited by 1,610
  2. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,609
  3. 2020Rethinking Semantic Segmentation from a Sequence-to-Sequence Perspective with Transformerscited by 1,368
  4. 2022VideoMAE: Masked Autoencoders are Data-Efficient Learners for Self-Supervised Video Pre-Trainingcited by 1,305
  5. 2017Random Erasing Data Augmentationcited by 1,235
  6. 2017A Closer Look at Spatiotemporal Convolutions for Action Recognitioncited by 1,139
  7. 2017The "something something" video database for learning and evaluating visual common sensecited by 1,092
  8. 2024Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Modelcited by 1,051
  9. 2024VMamba: Visual State Space Modelcited by 955
  10. 2021Multiscale Vision Transformerscited by 896
  11. 2020Pre-Trained Image Processing Transformercited by 891
  12. 2018TSM: Temporal Shift Module for Efficient Video Understandingcited by 852
  13. 2022Simplified State Space Layers for Sequence Modelingcited by 776
  14. 2021Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layerscited by 607
  15. 2017Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classificationcited by 521
  16. 2023VideoMAE V2: Scaling Video Masked Autoencoders with Dual Maskingcited by 516
  17. 2022Masked Autoencoders As Spatiotemporal Learnerscited by 428
  18. 2021VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Textcited by 427
  19. 2017Learning Spatio-Temporal Representation with Pseudo-3D Residual Networkscited by 386
  20. 2020Spatiotemporal Contrastive Video Representation Learningcited by 372
  21. 2022Diagonal State Spaces are as Effective as Structured State Spacescited by 350
  22. 2021Video Transformer Networkcited by 271
  23. 2018Long-Term Feature Banks for Detailed Video Understandingcited by 239
  24. 2021Transformer Meets Tracker: Exploiting Temporal Context for Robust Visual Trackingcited by 220
  25. 2017Learning Spatio-Temporal Features with 3D Residual Networks for Action Recognitioncited by 178
  26. 2020SpeedNet: Learning the Speediness in Videoscited by 171
  27. 2021Towards Long-Form Video Understandingcited by 146
  28. 2020Memory-augmented Dense Predictive Coding for Video Representation Learningcited by 137
  29. 2021MoViNets: Mobile Video Networks for Efficient Video Recognitioncited by 135
  30. 2020Disentangled Non-Local Neural Networkscited by 132
  31. 2022Long Movie Clip Classification with State-Space Video Modelscited by 106
  32. 2020Actor-Transformers for Group Activity Recognitioncited by 84
  33. 2024Mamba-ND: Selective State Space Modeling for Multi-Dimensional Datacited by 84
  34. 2020Video Representation Learning with Visual Tempo Consistencycited by 57
  35. 2019Lightweight Network Architecture for Real-Time Action Recognitioncited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 13; 2026: 6.

25 papers.

Sort:

By area:

  1. 2025STORM: Token-Efficient Long Video Understanding for Multimodal LLMscited by 46
  2. 2024A Comprehensive Survey of Action Quality Assessment: Method and Benchmarkcited by 20
  3. 2025AuroraLong: Bringing RNNs Back to Efficient Open-Ended Video Understandingcited by 9
  4. 2024Optical Flow Representation Alignment Mamba Diffusion Model for Medical Video Generationcited by 7
  5. 2025Moment Sampling in Video LLMs for Long-Form Video QAcited by 5
  6. 2024Parameter Efficient Mamba Tuning via Projector-targeted Diagonal-centric Linear Transformationcited by 4
  7. 2025High-Resolution Spatiotemporal Modeling with Global-Local State Space Models for Video-Based Human Pose Estimationcited by 4
  8. 2025MVQA: Mamba with Unified Sampling for Efficient Video Quality Assessmentcited by 2
  9. 2025InterMamba: Efficient Human-Human Interaction Generation with Adaptive Spatio-Temporal Mambacited by 2
  10. 2024Identifying Spatio-Temporal Drivers of Extreme Eventscited by 1
  11. 2024Empathetic Response in Audio-Visual Conversations Using Emotion Preference Optimization and MambaCompressorcited by 1
  12. 2025BrainMT: A Hybrid Mamba-Transformer Architecture for Modeling Long-Range Dependencies in Functional MRI Datacited by 1
13 more
  1. 2025HieraMamba: Video Temporal Grounding via Hierarchical Anchor-Mamba Poolingcited by 1
  2. 2026RS-SSM: Refining Forgotten Specifics in State Space Model for Video Semantic Segmentationcited by 1
  3. 2026HiCrew: Hierarchical Reasoning for Long-Form Video Understanding via Question-Aware Multi-Agent Collaborationcited by 1
  4. 2024PASS: Path-selective State Space Model for Event-based Recognitioncited by 0
  5. 2025ReynoldsFlow: Physics-Inspired Spatiotemporal Flow Representation for Video Understandingcited by 0
  6. 2025Mamba-3D as Masked Autoencoders for Accurate and Data-Efficient Analysis of Medical Ultrasound Videoscited by 0
  7. 2025Towards Efficient Vision State Space Models via Token Mergingcited by 0
  8. 2025Time-Scaling State-Space Models for Dense Video Captioningcited by 0
  9. 2025Beyond Real versus Fake Towards Intent-Aware Video Analysiscited by 0
  10. 2026Geometric Transformation-Embedded Mamba for Learned Video Compressioncited by 0
  11. 2026TrajMamba: An Ego-Motion-Guided Mamba Model for Pedestrian Trajectory Prediction from an Egocentric Perspectivecited by 0
  12. 2026Stitch-Inferencer: Enhance Endoscopic Video Segmentation and Tracking via Panoramic Reconstructioncited by 0
  13. 2026Why Is Video Still So Expensive? A Survey of Inference-Efficiency Mechanisms in Video and Audiovisual LLMscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024VideoMamba: State Space Model for Efficient Video Understanding
  2. 2024Snakes and Ladders: Two Steps Up for VideoMamba
  3. 2025MoMa: Modulating Mamba for Adapting Image Foundation Models to Video Recognition
  4. 2024Video Mamba Suite: State Space Model as a Versatile Alternative for Video Understanding
  5. 2025A Survey on Mamba Architecture for Vision Applications
  6. 2024A Survey on Visual Mamba
  7. 2026VideoSEMA: a scalable and efficient Mamba-like attention for video understanding
  8. 2024PlainMamba: Improving Non-Hierarchical Mamba in Visual Recognition