mlsanityData to 7 Oct 2026

LocalMamba: Visual State Space Model with Windowed Selective Scan

arXiv 2403.09338 ↗First posted 14 Mar 2024cs.CV cs.AIcited by 235

On the map: State-space sequence models (S4, Mamba) → Vision Mamba backbones

Abstract (arXiv)

Recent advancements in state space models, notably Mamba, have demonstrated significant progress in modeling long sequences for tasks like language understanding. Yet, their application in vision tasks has not markedly surpassed the performance of traditional Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs). This paper posits that the key to enhancing Vision Mamba (ViM) lies in optimizing scan directions for sequence modeling. Traditional ViM approaches, which flatten spatial tokens, overlook the preservation of local 2D dependencies, thereby elongating the distance between adjacent tokens. We introduce a novel local scanning strategy that divides images into distinct windows, effectively capturing local dependencies while maintaining a global perspective. Additionally, acknowledging the varying preferences for scan patterns across different network layers, we propose a dynamic method to independently search for the optimal scan choices for each layer, substantially improving performance. Extensive experiments across both plain and hierarchical models underscore our approach's superiority in effectively capturing image representations. For example, our model significantly outperforms Vim-Ti by 3.1% on ImageNet with the same 1.5G FLOPs. Code is available at: https://github.com/hunto/LocalMamba.

Key facts. Cited by 235 papers in the map (data through 7 Oct 2026). Placed in Vision Mamba backbones, one of 10 directions in State-space sequence models (S4, Mamba). Builds on 50 of its 58 references that are in the map. First posted on arXiv 14 Mar 2024.

Builds on

50 of 58 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model17 Jan

Other references · 49

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  6. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  7. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  8. 2017Mask R-CNNcited by 7,259
  9. 2022Scalable Diffusion Models with Transformerscited by 5,931
  10. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  11. 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
  12. 2020Training data-efficient image transformers & distillation through attentioncited by 4,560
37 more
  1. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  2. 2016Aggregated Residual Transformations for Deep Neural Networkscited by 2,540
  3. 2021Efficiently Modeling Long Sequences with Structured State Spacescited by 2,424
  4. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  5. 2018DARTS: Differentiable Architecture Searchcited by 1,782
  6. 2019MMDetection: Open MMLab Detection Toolbox and Benchmarkcited by 1,379
  7. 2018Unified Perceptual Parsing for Scene Understandingcited by 1,265
  8. 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,147
  9. 2016Semantic Understanding of Scenes through the ADE20K Datasetcited by 1,063
  10. 2024VMamba: Visual State Space Modelcited by 955
  11. 2020Designing Network Design Spacescited by 954
  12. 2021Segmenter: Transformer for Semantic Segmentationcited by 855
  13. 2022Simplified State Space Layers for Sequence Modelingcited by 776
  14. 2022Exploring Plain Vision Transformer Backbones for Object Detectioncited by 723
  15. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 666
  16. 2021Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layerscited by 607
  17. 2021Twins: Revisiting the Design of Spatial Attention in Vision Transformerscited by 565
  18. 2022On the Parameterization and Initialization of Diagonal State Space Modelscited by 418
  19. 2022DeiT III: Revenge of the ViTcited by 417
  20. 2024U-Mamba: Enhancing Long-range Dependency for Biomedical Image Segmentationcited by 412
  21. 2019Single Path One-Shot Neural Architecture Search with Uniform Samplingcited by 411
  22. 2023Resurrecting Recurrent Neural Networks for Long Sequencescited by 397
  23. 2022Diagonal State Spaces are as Effective as Structured State Spacescited by 350
  24. 2024VM-UNet: Vision Mamba UNet for Medical Image Segmentationcited by 286
  25. 2024SegMamba: Long-range Sequential Modeling Mamba For 3D Medical Image Segmentationcited by 266
  26. 2019PC-DARTS: Partial Channel Connections for Memory-Efficient Architecture Searchcited by 265
  27. 2022Long Range Language Modeling via Gated State Spacescited by 265
  28. 2024Swin-UMamba: Mamba-based UNet with ImageNet-based pretrainingcited by 179
  29. 2021You Only Look at One Sequence: Rethinking Transformer in Vision through Object Detectioncited by 170
  30. 2021StyleSwin: Transformer-based GAN for High-resolution Image Generationcited by 152
  31. 2021ViTGAN: Training GANs with Vision Transformerscited by 110
  32. 2022What Makes Convolutional Models Great on Long Sequence Modeling?cited by 86
  33. 2020GreedyNAS: Towards Fast One-Shot NAS with Greedy Supernetcited by 69
  34. 2022Temporally Efficient Vision Transformer for Video Instance Segmentationcited by 55
  35. 2022TokenMix: Rethinking Image Mixing for Data Augmentation in Vision Transformerscited by 48
  36. 2022LightViT: Towards Light-Weight Convolution-Free Vision Transformerscited by 40
  37. 2022GhostNets on Heterogeneous Devices via Cheap Operationscited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 88; 2025: 89; 2026: 58.

235 papers.

Sort:

By area:

  1. 2024MambaOut: Do We Really Need Mamba for Vision?cited by 142
  2. 2024PlainMamba: Improving Non-Hierarchical Mamba in Visual Recognitioncited by 141
  3. 2024MambaAD: Exploring State Space Models for Multi-class Unsupervised Anomaly Detectioncited by 113
  4. 2024Long-LRM: Long-sequence Large Reconstruction Model for Wide-coverage Gaussian Splatscited by 111
  5. 2024Demystify Mamba in Vision: A Linear Attention Perspectivecited by 101
  6. 2024Mamba-360: Survey of State Space Models as Transformer Alternative for Long Sequence Modelling: Methods, Applications, and Challengescited by 63
  7. 2024Multi-Scale VMamba: Hierarchy in Hierarchy Visual State Space Modelcited by 62
  8. 2024State Space Model for New-Generation Network Alternative to Transformers: A Surveycited by 59
  9. 2024FreqMamba: Viewing Mamba from a Frequency Perspective for Image Derainingcited by 57
  10. 2024RainMamba: Enhanced Locality Learning with State Space Models for Video Derainingcited by 56
  11. 2024Vision Mamba: A Comprehensive Survey and Taxonomycited by 53
  12. 2024A Survey on Visual Mambacited by 50
38 more
  1. 2024MambaLLIE: Implicit Retinex-Aware Low Light Enhancement with Global-then-Local State Spacecited by 48
  2. 2024CDMamba: Incorporating Local Clues into Mamba for Remote Sensing Image Binary Change Detectioncited by 48
  3. 2024MaIR: A Locality- and Continuity-Preserving Mamba for Image Restorationcited by 43
  4. 2024Mamba-R: Vision Mamba ALSO Needs Registerscited by 41
  5. 2024MambaMixer: Efficient Selective State Space Models with Dual Token and Channel Selectioncited by 40
  6. 2024Spatial-Mamba: Effective Visual State Space Models via Structure-aware State Fusioncited by 40
  7. 2024A Survey of Mambacited by 39
  8. 20242DMamba: Efficient State Space Model for Image Representation with Applications on Giga-Pixel Whole Slide Image Classificationcited by 35
  9. 2024VSSD: Vision Mamba with Non-Causal State Space Dualitycited by 29
  10. 2025Scaling Laws in Patchification: An Image Is Worth 50,176 Tokens And Morecited by 28
  11. 2025MambaQuant: Quantizing the Mamba Family with Variance Aligned Rotation Methodscited by 26
  12. 2024MVGamba: Unify 3D Content Generation as State Space Sequence Modelingcited by 23
  13. 2024Autoregressive Pretraining with Mamba in Visioncited by 21
  14. 2024Mamba in Vision: A Comprehensive Survey of Techniques and Applicationscited by 18
  15. 2024QuadMamba: Learning Quadtree-based Selective Scan for Visual State Space Modelcited by 18
  16. 2024COMO: Cross-Mamba Interaction and Offset-Guided Fusion for Multimodal Object Detectioncited by 18
  17. 2025MambaIC: State Space Models for High-Performance Learned Image Compressioncited by 18
  18. 2025DefMamba: Deformable Visual State Space Modelcited by 18
  19. 2024EfficientViM: Efficient Vision Mamba with Hidden State Mixer based State Space Dualitycited by 17
  20. 2024A Comprehensive Survey of Mamba Architectures for Medical Image Analysis: Classification, Segmentation, Restoration and Beyondcited by 16
  21. 2024MobileMamba: Lightweight Multi-Receptive Visual Mamba Networkcited by 16
  22. 2024GroupMamba: Efficient Group-Based Visual State Space Modelcited by 15
  23. 2024TinyViM: Frequency Decoupling for Tiny Hybrid Vision Mambacited by 15
  24. 2025AVS-Mamba: Exploring Temporal and Multi-modal Mamba for Audio-Visual Segmentationcited by 15
  25. 2025ViT3: Unlocking Test-Time Training in Visioncited by 15
  26. 2024MambaCSR: Dual-Interleaved Scanning for Compressed Image Super-Resolution With SSMscited by 14
  27. 2024MambaTS: Improved Selective State Space Models for Long-term Time Series Forecastingcited by 13
  28. 2024ViG: Linear-complexity Visual Sequence Learning with Gated Linear Attentioncited by 13
  29. 2024GrootVL: Tree Topology is All You Need in State Space Modelcited by 13
  30. 2024Understanding Robustness of Visual State Space Models for Image Classificationcited by 12
  31. 2024Scalable Visual State Space Model with Fractal Scanningcited by 11
  32. 2024IGroupSS-Mamba: Interval Group Spatial-Spectral Mamba for Hyperspectral Image Classificationcited by 11
  33. 2024DiMSUM: Diffusion Mamba -- A Scalable and Unified Spatial-Frequency Method for Image Generationcited by 11
  34. 2024MambaLCT: Boosting Tracking via Long-term Context State Space Modelcited by 11
  35. 2024ML-Mamba: Efficient Multi-Modal Large Language Model Utilizing Mamba-2cited by 10
  36. 2025UniMamba: Unified Spatial-Channel Representation Learning with Group-Efficient Mamba for LiDAR-based 3D Object Detectioncited by 10
  37. 2025CWNet: Causal Wavelet Network for Low-Light Image Enhancementcited by 10
  38. 2024CoMamba: Real-time Cooperative Perception Unlocked with State Space Modelscited by 9

50 most cited of 235 All 235 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Visual Mamba: A Survey and New Outlooks
  2. 2024EfficientVMamba: Atrous Selective Scan for Light Weight Visual Mamba
  3. 2025VCMamba: Bridging Convolutions with Multi-Directional Mamba for Efficient Visual Representation
  4. 2026SF-Mamba: Rethinking State Space Model for Vision
  5. 2025Visual Attention Exploration in Vision-Based Mamba Models
  6. 2024A Novel State Space Model with Local Enhancement and State Sharing for Image Fusion
  7. 2024Video Mamba Suite: State Space Model as a Versatile Alternative for Video Understanding
  8. 2024Computation-Efficient Era: A Comprehensive Survey of State Space Models in Medical Image Analysis