# Spatial-Mamba: Effective Visual State Space Models via Structure-aware State Fusion

arXiv: https://arxiv.org/abs/2410.15091 · HTML: https://mlsanity.com/paper/2410.15091/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Placed in Vision Mamba backbones, one of 10 directions in State-space sequence models (S4, Mamba).
- Builds on 47 of its 57 references that are in the map.
- First posted on arXiv 19 Oct 2024.

## Placed in

- State-space sequence models (S4, Mamba) / [Vision Mamba backbones](https://mlsanity.com/ssm/vision-mamba-backbones/)

## Builds on (most cited first, 30 of 47)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [Transformers are RNNs: Fast Autoregressive Transformers with Linear Attention](https://mlsanity.com/paper/2006.16236/) (2020)
- [Dream to Control: Learning Behaviors by Latent Imagination](https://mlsanity.com/paper/1912.01603/) (2019)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [MMDetection: Open MMLab Detection Toolbox and Benchmark](https://mlsanity.com/paper/1906.07155/) (2019)
- [Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality](https://mlsanity.com/paper/2405.21060/) (2024)
- [Cascade R-CNN: Delving into High Quality Object Detection](https://mlsanity.com/paper/1712.00726/) (2017)
- [Unified Perceptual Parsing for Scene Understanding](https://mlsanity.com/paper/1807.10221/) (2018)
- [Multi-Scale Context Aggregation by Dilated Convolutions](https://mlsanity.com/paper/1511.07122/) (2015)
- [Semantic Understanding of Scenes through the ADE20K Dataset](https://mlsanity.com/paper/1608.05442/) (2016)
- [Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model](https://mlsanity.com/paper/2401.09417/) (2024)
- [VMamba: Visual State Space Model](https://mlsanity.com/paper/2401.10166/) (2024)
- [Designing Network Design Spaces](https://mlsanity.com/paper/2003.13678/) (2020)
- [Simplified State Space Layers for Sequence Modeling](https://mlsanity.com/paper/2208.04933/) (2022)
- [PVT v2: Improved Baselines with Pyramid Vision Transformer](https://mlsanity.com/paper/2106.13797/) (2021)
- [Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layers](https://mlsanity.com/paper/2110.13985/) (2021)
- [HiPPO: Recurrent Memory with Optimal Polynomial Projections](https://mlsanity.com/paper/2008.07669/) (2020)
- [Hungry Hungry Hippos: Towards Language Modeling with State Space Models](https://mlsanity.com/paper/2212.14052/) (2022)
- [U-Mamba: Enhancing Long-range Dependency for Biomedical Image Segmentation](https://mlsanity.com/paper/2401.04722/) (2024)
- [Dynamic Convolution: Attention over Convolution Kernels](https://mlsanity.com/paper/1912.03458/) (2019)
- [MambaIR: A Simple Baseline for Image Restoration with State-Space Model](https://mlsanity.com/paper/2402.15648/) (2024)
- [Diagonal State Spaces are as Effective as Structured State Spaces](https://mlsanity.com/paper/2203.14343/) (2022)
- [VideoMamba: State Space Model for Efficient Video Understanding](https://mlsanity.com/paper/2403.06977/) (2024)

## Built on by (most cited first, 30 of 41)

- [Frequency-Dynamic Attention Modulation for Dense Prediction](https://mlsanity.com/paper/2507.12006/) (2025)
- [A Survey on Mamba Architecture for Vision Applications](https://mlsanity.com/paper/2502.07161/) (2025)
- [TransMamba: A Sequence-Level Hybrid Transformer-Mamba Language Model](https://mlsanity.com/paper/2503.24067/) (2025)
- [UIS-Mamba: Exploring Mamba for Underwater Instance Segmentation via Dynamic Tree Scan and Hidden State Weaken](https://mlsanity.com/paper/2508.00421/) (2025)
- [SF-Mamba: Rethinking State Space Model for Vision](https://mlsanity.com/paper/2603.16423/) (2026)
- [DPMambaIR: All-in-One Image Restoration via Degradation-Aware Prompt State Space Model](https://mlsanity.com/paper/2504.17732/) (2025)
- [Spatial Frequency Modulation for Semantic Segmentation](https://mlsanity.com/paper/2507.11893/) (2025)
- [LIDAR: Lightweight Adaptive Cue-Aware Fusion Vision Mamba for Multimodal Segmentation of Structural Cracks](https://mlsanity.com/paper/2507.22477/) (2025)
- [A Controlled Benchmark of Visual State-Space Backbones with Domain-Shift and Boundary Analysis for Remote-Sensing Segmentation](https://mlsanity.com/paper/2604.18721/) (2026)
- [GPSMamba: A Global Phase and Spectral Prompt-guided Mamba for Infrared Image Super-Resolution](https://mlsanity.com/paper/2507.18998/) (2025)
- [DiM-TS: Bridge the Gap between Selective State Space Models and Time Series for Generative Modeling](https://mlsanity.com/paper/2511.18312/) (2025)
- [O3N: Omnidirectional Open-Vocabulary Occupancy Prediction for Embodied Intelligent Robotics](https://mlsanity.com/paper/2603.12144/) (2026)
- [Deformba: Vision State Space Model with Adaptive State Fusion](https://mlsanity.com/paper/2605.21308/) (2026)
- [FractalMamba++: Scaling Vision Mamba Across Resolutions via Hilbert Fractal Geometry](https://mlsanity.com/paper/2505.14062/) (2025)
- [Polyline Path Masked Attention for Vision Transformer](https://mlsanity.com/paper/2506.15940/) (2025)
- [A2Mamba: Attention-augmented State Space Models for Visual Recognition](https://mlsanity.com/paper/2507.16624/) (2025)
- [Beyond Illumination: Fine-Grained Detail Preservation in Extreme Dark Image Restoration](https://mlsanity.com/paper/2508.03336/) (2025)
- [LISM: Long-range Integrative State space Models via Input-Latent State Interactions](https://mlsanity.com/paper/2509.04226/) (2025)
- [MSF-Mamba: Motion-aware State Fusion Mamba for Efficient Micro-Gesture Recognition](https://mlsanity.com/paper/2510.10478/) (2025)
- [ChessMamba: Structure-Aware Interleaving of State Spaces for Change Detection in Remote Sensing Images](https://mlsanity.com/paper/2511.19882/) (2025)
- [CellMamba: Adaptive Mamba for Accurate and Efficient Cell Detection](https://mlsanity.com/paper/2512.21803/) (2025)
- [SMTrack: State-Aware Mamba for Efficient Temporal Modeling in Visual Tracking](https://mlsanity.com/paper/2602.01677/) (2026)
- [Rotation Equivariant Mamba for Vision Tasks](https://mlsanity.com/paper/2603.09138/) (2026)
- [RS-SSM: Refining Forgotten Specifics in State Space Model for Video Semantic Segmentation](https://mlsanity.com/paper/2603.24295/) (2026)
- [Vision Non-Causal Trapezoidal Mamba: Eliminating Directional Scanning in Vision SSMs with Second-Order Dynamics](https://mlsanity.com/paper/2607.03589/) (2026)
- [Rethinking Convergence in Deep Learning: The Predictive-Corrective Paradigm for Anatomy-Informed Brain MRI Segmentation](https://mlsanity.com/paper/2510.15439/) (2025)
- [Deltat-Mamba3D: A Time-Aware Spatio-Temporal State-Space Model for Breast Cancer Risk Prediction](https://mlsanity.com/paper/2510.19003/) (2025)
- [A Spatial Semantics and Continuity Perception Attention for Remote Sensing Water Body Change Detection](https://mlsanity.com/paper/2511.16143/) (2025)
- [DF-Mamba: Deformable State Space Modeling for 3D Hand Pose Estimation in Interactions](https://mlsanity.com/paper/2512.02727/) (2025)
- [LinMU: Multimodal Understanding Made Linear](https://mlsanity.com/paper/2601.01322/) (2026)

## Abstract (arXiv)

Selective state space models (SSMs), such as Mamba, highly excel at capturing long-range dependencies in 1D sequential data, while their applications to 2D vision tasks still face challenges. Current visual SSMs often convert images into 1D sequences and employ various scanning patterns to incorporate local spatial dependencies. However, these methods are limited in effectively capturing the complex image spatial structures and the increased computational cost caused by the lengthened scanning paths. To address these limitations, we propose Spatial-Mamba, a novel approach that establishes neighborhood connectivity directly in the state space. Instead of relying solely on sequential state transitions, we introduce a structure-aware state fusion equation, which leverages dilated convolutions to capture image spatial structural dependencies, significantly enhancing the flow of visual contextual information. Spatial-Mamba proceeds in three stages: initial state computation in a unidirectional scan, spatial context acquisition through structure-aware state fusion, and final state computation using the observation equation. Our theoretical analysis shows that Spatial-Mamba unifies the original Mamba and linear attention under the same matrix multiplication framework, providing a deeper understanding of our method. Experimental results demonstrate that Spatial-Mamba, even with a single scan, attains or surpasses the state-of-the-art SSM-based models in image classification, detection and segmentation. Source codes and trained models can be found at https://github.com/EdwardChasel/Spatial-Mamba.