mlsanityData to 7 Oct 2026

MambaOut: Do We Really Need Mamba for Vision?

arXiv 2405.07992 ↗First posted 13 May 2024cs.CV cs.AI cs.LGcited by 142

On the map: State-space sequence models (S4, Mamba) → Vision Mamba backbones

Abstract (arXiv)

Mamba, an architecture with RNN-like token mixer of state space model (SSM), was recently introduced to address the quadratic complexity of the attention mechanism and subsequently applied to vision tasks. Nevertheless, the performance of Mamba for vision is often underwhelming when compared with convolutional and attention-based models. In this paper, we delve into the essence of Mamba, and conceptually conclude that Mamba is ideally suited for tasks with long-sequence and autoregressive characteristics. For vision tasks, as image classification does not align with either characteristic, we hypothesize that Mamba is not necessary for this task; Detection and segmentation tasks are also not autoregressive, yet they adhere to the long-sequence characteristic, so we believe it is still worthwhile to explore Mamba's potential for these tasks. To empirically verify our hypotheses, we construct a series of models named MambaOut through stacking Mamba blocks while removing their core token mixer, SSM. Experimental results strongly support our hypotheses. Specifically, our MambaOut model surpasses all visual Mamba models on ImageNet image classification, indicating that Mamba is indeed unnecessary for this task. As for detection and segmentation, MambaOut cannot match the performance of state-of-the-art visual Mamba models, demonstrating the potential of Mamba for long-sequence visual tasks. The code is available at https://github.com/yuweihao/MambaOut

Key facts. Cited by 142 papers in the map (data through 7 Oct 2026). Placed in Vision Mamba backbones, one of 10 directions in State-space sequence models (S4, Mamba). Builds on 90 of its 111 references that are in the map. First posted on arXiv 13 May 2024.

Builds on

90 of 111 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024A Survey on Visual Mamba24 Apr
  2. 2024PlainMamba: Improving Non-Hierarchical Mamba in Visual Recognition26 Mar
  3. 2024SiMBA: Simplified Mamba-Based Architecture for Vision and Multivariate Time series22 Mar
  4. 2024EfficientVMamba: Atrous Selective Scan for Light Weight Visual Mamba15 Mar
  5. 2024LocalMamba: Visual State Space Model with Windowed Selective Scan14 Mar
  6. 2024Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model17 Jan

Other references · 84

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2020Language Models are Few-Shot Learnerscited by 32,199
  5. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  6. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  7. 2023GPT-4 Technical Reportcited by 18,132
  8. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  9. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  10. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  11. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  12. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
38 more
  1. 2017Mask R-CNNcited by 7,259
  2. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  3. 2016Layer Normalizationcited by 5,579
  4. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  5. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  6. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  7. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  8. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
  9. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  10. 2022A ConvNet for the 2020scited by 3,545
  11. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  12. 2020Longformer: The Long-Document Transformercited by 3,362
  13. 2019CutMix: Regularization Strategy to Train Strong Classifiers with Localizable Featurescited by 2,875
  14. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
  15. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  16. 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,320
  17. 2019RandAugment: Practical automated data augmentation with a reduced search spacecited by 2,261
  18. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 1,979
  19. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
  20. 2020Big Bird: Transformers for Longer Sequencescited by 1,673
  21. 2018Group Normalizationcited by 1,514
  22. 2020Linformer: Self-Attention with Linear Complexitycited by 1,462
  23. 2019MMDetection: Open MMLab Detection Toolbox and Benchmarkcited by 1,379
  24. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,265
  25. 2017Random Erasing Data Augmentationcited by 1,235
  26. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,132
  27. 2021Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNetcited by 1,096
  28. 2016Deep Networks with Stochastic Depthcited by 973
  29. 2024VMamba: Visual State Space Modelcited by 955
  30. 2023RWKV: Reinventing RNNs for the Transformer Eracited by 814
  31. 2020Efficient Transformers: A Surveycited by 784
  32. 2021Going deeper with Image Transformerscited by 759
  33. 2021PVT v2: Improved Baselines with Pyramid Vision Transformercited by 749
  34. 2019Compressive Transformers for Long-Range Sequence Modellingcited by 699
  35. 2016Language Modeling with Gated Convolutional Networkscited by 688
  36. 2019Attention is not not Explanationcited by 629
  37. 2021Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layerscited by 607
  38. 2021CoAtNet: Marrying Convolution and Attention for All Data Sizescited by 566

50 most cited of 84 All 84 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 28; 2025: 68; 2026: 46.

142 papers.

Sort:

By area:

  1. 2024Demystify Mamba in Vision: A Linear Attention Perspectivecited by 101
  2. 2024Spatial-Mamba: Effective Visual State Space Models via Structure-aware State Fusioncited by 40
  3. 2025VMoBA: Mixture-of-Block Attention for Video Diffusion Modelscited by 35
  4. 2024Touchstone Benchmark: Are We on the Right Way for Evaluating AI Algorithms for Medical Segmentation?cited by 31
  5. 2024VSSD: Vision Mamba with Non-Causal State Space Dualitycited by 29
  6. 2025StableCodec: Taming One-Step Diffusion for Extreme Image Compressioncited by 29
  7. 2025UAVScenes: A Multi-Modal Dataset for UAVscited by 26
  8. 2024MVGamba: Unify 3D Content Generation as State Space Sequence Modelingcited by 23
  9. 2024InfiniMotion: Mamba Boosts Memory in Transformer for Arbitrary Long Motion Generationcited by 23
  10. 2024MobileMamba: Lightweight Multi-Receptive Visual Mamba Networkcited by 16
  11. 2024DyG-Mamba: Continuous State Space Modeling on Dynamic Graphscited by 15
  12. 2024Robust Tracking via Mamba-based Context-aware Token Learningcited by 15
38 more
  1. 2025ViT3: Unlocking Test-Time Training in Visioncited by 15
  2. 2024SparX: A Sparse Cross-Layer Connection Mechanism for Hierarchical Vision Mamba and Transformer Networkscited by 14
  3. 2024SMILES-Mamba: Chemical Mamba Foundation Models for Drug ADMET Predictioncited by 13
  4. 2024IRSRMamba: Infrared Image Super-Resolution via Mamba-based Wavelet Transform Feature Modulation Modelcited by 12
  5. 2025Frequency-Dynamic Attention Modulation for Dense Predictioncited by 10
  6. 2025DAMamba: Vision State Space Model with Dynamic Adaptive Scancited by 9
  7. 2025Artificial Hippocampus Networks for Efficient Long-Context Modelingcited by 9
  8. 2025Semi-Supervised State-Space Model with Dynamic Stacking Filter for Real-World Video Derainingcited by 8
  9. 2024Is Mamba Compatible with Trajectory Optimization in Offline Reinforcement Learning?cited by 7
  10. 2024MUSE: Mamba is Efficient Multi-scale Learner for Text-video Retrievalcited by 7
  11. 2026ZACH-ViT: Regime-Dependent Inductive Bias in Compact Vision Transformers for Medical Imagingcited by 7
  12. 2024A Hybrid Transformer-Mamba Network for Single Image Derainingcited by 6
  13. 2025TransMamba: Fast Universal Architecture Adaption from Transformers to Mambacited by 6
  14. 2024Seg-LSTM: Performance of xLSTM for Semantic Segmentation of Remotely Sensed Imagescited by 5
  15. 2024LibraGrad: Balancing Gradient Flow for Universally Better Vision Transformer Attributionscited by 5
  16. 2024EMOv2: Pushing 5M Vision Model Frontiercited by 5
  17. 2025ViT-Linearizer: Distilling Quadratic Knowledge into Linear-Time Vision Modelscited by 5
  18. 2024Protein-Mamba: Biological Mamba Models for Protein Function Predictioncited by 4
  19. 2025Parallel Sequence Modeling via Generalized Spatial Propagation Networkcited by 4
  20. 2025After the Party: Navigating the Mapping From Color to Ambient Lightingcited by 4
  21. 2025VCMamba: Bridging Convolutions with Multi-Directional Mamba for Efficient Visual Representationcited by 4
  22. 2025TimeViper: A Hybrid Mamba-Transformer Vision-Language Model for Efficient Long Video Understandingcited by 4
  23. 2025MambaTAD: When State-Space Models Meet Long-Range Temporal Action Detectioncited by 4
  24. 2025Ultra-Low Bitrate Perceptual Image Compression with Shallow Encodercited by 4
  25. 2026SF-Mamba: Rethinking State Space Model for Visioncited by 4
  26. 2024Mamba2D: A Natively Multi-Dimensional State-Space Model for Vision Taskscited by 3
  27. 2025Revisiting Convolution Architecture in the Realm of DNA Foundation Modelscited by 3
  28. 2025Gaze-Guided Learning: Avoiding Shortcut Bias in Visual Classificationcited by 3
  29. 2025S2AFormer: Strip Self-Attention for Efficient Vision Transformercited by 3
  30. 2025MUG: Pseudo Labeling Augmented Audio-Visual Mamba Network for Audio-Visual Video Parsingcited by 3
  31. 2025LIDAR: Lightweight Adaptive Cue-Aware Fusion Vision Mamba for Multimodal Segmentation of Structural Crackscited by 3
  32. 2025SAGA: Selective Adaptive Gating for Efficient and Expressive Linear Attentioncited by 3
  33. 2026EMFormer: Efficient Multi-Scale Transformer for Accumulative Context Weather Forecastingcited by 3
  34. 2026HAMSA: Scanning-Free Vision State Space Models via SpectralPulseNetcited by 3
  35. 2024miMamba: EEG-based Emotion Recognition with Multi-scale Inverted Mamba Modelscited by 2
  36. 2025MambaFlow: A Mamba-Centric Architecture for End-to-End Optical Flow Estimationcited by 2
  37. 2025MVQA: Mamba with Unified Sampling for Efficient Video Quality Assessmentcited by 2
  38. 2025Selective Structured State Space for Multispectral-fused Small Target Detectioncited by 2

50 most cited of 142 All 142 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Visual Mamba: A Survey and New Outlooks
  2. 2024Mamba in Vision: A Comprehensive Survey of Techniques and Applications
  3. 2024Video Mamba Suite: State Space Model as a Versatile Alternative for Video Understanding
  4. 2025Dynamic Vision Mamba
  5. 2025A Survey on Mamba Architecture for Vision Applications
  6. 2024Autoregressive Pretraining with Mamba in Vision
  7. 2025Fast Vision Mamba: Pooling Spatial Dimensions for Accelerated Processing
  8. 2026MambaPSA: A Mamba-based Replacement for C2PSA in YOLO26