# DefMamba: Deformable Visual State Space Model

arXiv: https://arxiv.org/abs/2504.05794 · HTML: https://mlsanity.com/paper/2504.05794/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Placed in Vision Mamba backbones, one of 10 directions in State-space sequence models (S4, Mamba).
- Builds on 37 of its 48 references that are in the map.
- First posted on arXiv 8 Apr 2025.

## Placed in

- State-space sequence models (S4, Mamba) / [Vision Mamba backbones](https://mlsanity.com/ssm/vision-mamba-backbones/)

## Builds on (most cited first, 30 of 37)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [Unified Perceptual Parsing for Scene Understanding](https://mlsanity.com/paper/1807.10221/) (2018)
- [Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model](https://mlsanity.com/paper/2401.09417/) (2024)
- [Designing Network Design Spaces](https://mlsanity.com/paper/2003.13678/) (2020)
- [PVT v2: Improved Baselines with Pyramid Vision Transformer](https://mlsanity.com/paper/2106.13797/) (2021)
- [Deformable ConvNets v2: More Deformable, Better Results](https://mlsanity.com/paper/1811.11168/) (2018)
- [CoAtNet: Marrying Convolution and Attention for All Data Sizes](https://mlsanity.com/paper/2106.04803/) (2021)
- [GhostNet: More Features from Cheap Operations](https://mlsanity.com/paper/1911.11907/) (2019)
- [Vision Transformer with Deformable Attention](https://mlsanity.com/paper/2201.00520/) (2022)
- [SegMamba: Long-range Sequential Modeling Mamba For 3D Medical Image Segmentation](https://mlsanity.com/paper/2401.13560/) (2024)
- [Neighborhood Attention Transformer](https://mlsanity.com/paper/2204.07143/) (2022)
- [LocalMamba: Visual State Space Model with Windowed Selective Scan](https://mlsanity.com/paper/2403.09338/) (2024)
- [Swin-UMamba: Mamba-based UNet with ImageNet-based pretraining](https://mlsanity.com/paper/2402.03302/) (2024)
- [EfficientVMamba: Atrous Selective Scan for Light Weight Visual Mamba](https://mlsanity.com/paper/2403.09977/) (2024)
- [PlainMamba: Improving Non-Hierarchical Mamba in Visual Recognition](https://mlsanity.com/paper/2403.17695/) (2024)
- [CrossFormer: A Versatile Vision Transformer Hinging on Cross-scale Attention](https://mlsanity.com/paper/2108.00154/) (2021)
- [RSMamba: Remote Sensing Image Classification with State Space Model](https://mlsanity.com/paper/2403.19654/) (2024)
- [Conv2Former: A Simple Transformer-Style ConvNet for Visual Recognition](https://mlsanity.com/paper/2211.11943/) (2022)
- [Multi-Scale VMamba: Hierarchy in Hierarchy Visual State Space Model](https://mlsanity.com/paper/2405.14174/) (2024)

## Built on by (most cited first, 18 of 18)

- [SF-Mamba: Rethinking State Space Model for Vision](https://mlsanity.com/paper/2603.16423/) (2026)
- [SaFiRe: Saccade-Fixation Reiteration with Mamba for Referring Image Segmentation](https://mlsanity.com/paper/2510.10160/) (2025)
- [Deformba: Vision State Space Model with Adaptive State Fusion](https://mlsanity.com/paper/2605.21308/) (2026)
- [UINO-FSS: Unifying Representation Learning and Few-shot Segmentation via Hierarchical Distillation and Mamba-HyperCorrelation](https://mlsanity.com/paper/2504.15669/) (2025)
- [FractalMamba++: Scaling Vision Mamba Across Resolutions via Hilbert Fractal Geometry](https://mlsanity.com/paper/2505.14062/) (2025)
- [SEMA: a Scalable and Efficient Mamba like Attention via Token Localization and Averaging](https://mlsanity.com/paper/2506.08297/) (2025)
- [Partial Ring Scan: Revisiting Scan Order in Vision State Space Models](https://mlsanity.com/paper/2602.04170/) (2026)
- [GEM: Generating LiDAR World Model via Deformable Mamba](https://mlsanity.com/paper/2605.07326/) (2026)
- [Vision Non-Causal Trapezoidal Mamba: Eliminating Directional Scanning in Vision SSMs with Second-Order Dynamics](https://mlsanity.com/paper/2607.03589/) (2026)
- [Bend the Basics: Degradation-Aware Deformable Tokenization for All-in-One Image Restoration](https://mlsanity.com/paper/2608.06832/) (2026)
- [DM3D: Dynamic Mamba via Offset-Guided Feature Resampling for Point Cloud Understanding](https://mlsanity.com/paper/2512.03424/) (2025)
- [How to Build Robust, Scalable Models for GSV-Based Indicators in Neighborhood Research](https://mlsanity.com/paper/2601.06443/) (2026)
- [Frequency-Enhanced Hilbert Scanning Mamba for Short-Term Arctic Sea Ice Concentration Prediction](https://mlsanity.com/paper/2602.13522/) (2026)
- [Bridging the Geometry Mismatch: Frequency-Aware Anisotropic Serialization for Thin-Structure SSMs](https://mlsanity.com/paper/2603.28503/) (2026)
- [Can Graphs Help Vision SSMs See Better?](https://mlsanity.com/paper/2605.11300/) (2026)
- [AdaSurvMamba: Dynamic Fusion and Semantic Scanning for Multimodal Survival Analysis](https://mlsanity.com/paper/2607.16260/) (2026)
- [FU-Mamba: A Frequency-Enhanced Dynamic Scanning Framework for Oralscan Image Segmentation](https://mlsanity.com/paper/2608.26607/) (2026)
- [Token Clustering and Semantic Sequence Mamba for Hyperspectral Image Classification](https://mlsanity.com/paper/2609.28580/) (2026)

## Abstract (arXiv)

Recently, state space models (SSM), particularly Mamba, have attracted significant attention from scholars due to their ability to effectively balance computational efficiency and performance. However, most existing visual Mamba methods flatten images into 1D sequences using predefined scan orders, which results the model being less capable of utilizing the spatial structural information of the image during the feature extraction process. To address this issue, we proposed a novel visual foundation model called DefMamba. This model includes a multi-scale backbone structure and deformable mamba (DM) blocks, which dynamically adjust the scanning path to prioritize important information, thus enhancing the capture and processing of relevant input features. By combining a deformable scanning(DS) strategy, this model significantly improves its ability to learn image structures and detects changes in object details. Numerous experiments have shown that DefMamba achieves state-of-the-art performance in various visual tasks, including image classification, object detection, instance segmentation, and semantic segmentation. The code is open source on DefMamba.