# 2DMamba: Efficient State Space Model for Image Representation with Applications on Giga-Pixel Whole Slide Image Classification

arXiv: https://arxiv.org/abs/2412.00678 · HTML: https://mlsanity.com/paper/2412.00678/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Placed in Vision Mamba backbones, one of 10 directions in State-space sequence models (S4, Mamba).
- Builds on 31 of its 49 references that are in the map.
- First posted on arXiv 1 Dec 2024.

## Placed in

- State-space sequence models (S4, Mamba) / [Vision Mamba backbones](https://mlsanity.com/ssm/vision-mamba-backbones/)

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://mlsanity.com/paper/2307.08691/) (2023)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality](https://mlsanity.com/paper/2405.21060/) (2024)
- [Attention-based Deep Multiple Instance Learning](https://mlsanity.com/paper/1802.04712/) (2018)
- [Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model](https://mlsanity.com/paper/2401.09417/) (2024)
- [Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Images](https://mlsanity.com/paper/2004.09666/) (2020)
- [Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layers](https://mlsanity.com/paper/2110.13985/) (2021)
- [TransMIL: Transformer based Correlated Multiple Instance Learning for Whole Slide Image Classification](https://mlsanity.com/paper/2106.00908/) (2021)
- [Understanding the Effective Receptive Field in Deep Convolutional Neural Networks](https://mlsanity.com/paper/1701.04128/) (2017)
- [Dual-stream Multiple Instance Learning Network for Whole Slide Image Classification with Self-supervised Contrastive Learning](https://mlsanity.com/paper/2011.08939/) (2020)
- [Nystr"omformer: A Nystr"om-Based Algorithm for Approximating Self-Attention](https://mlsanity.com/paper/2102.03902/) (2021)
- [Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learning](https://mlsanity.com/paper/2206.02647/) (2022)
- [DTFD-MIL: Double-Tier Feature Distillation Multiple Instance Learning for Histopathology Whole Slide Image Classification](https://mlsanity.com/paper/2203.12081/) (2022)
- [LocalMamba: Visual State Space Model with Windowed Selective Scan](https://mlsanity.com/paper/2403.09338/) (2024)
- [EfficientVMamba: Atrous Selective Scan for Light Weight Visual Mamba](https://mlsanity.com/paper/2403.09977/) (2024)
- [BRACS: A Dataset for BReAst Carcinoma Subtyping in H&E Histology Images](https://mlsanity.com/paper/2111.04740/) (2021)
- [Multi-Scale VMamba: Hierarchy in Hierarchy Visual State Space Model](https://mlsanity.com/paper/2405.14174/) (2024)
- [Structured State Space Models for Multiple Instance Learning in Digital Pathology](https://mlsanity.com/paper/2306.15789/) (2023)
- [Computational Pathology: A Survey Review and The Way Forward](https://mlsanity.com/paper/2304.05482/) (2023)
- [Development and Evaluation of a Deep Neural Network for Histologic Classification of Renal Cell Carcinoma on Biopsy and Surgical Resection Slides](https://mlsanity.com/paper/2010.16380/) (2020)
- [Brain Diffuser: An End-to-End Brain Image to Brain Network Pipeline](https://mlsanity.com/paper/2303.06410/) (2023)
- [Prompt-MIL: Boosting Multi-Instance Learning Schemes via Task-specific Prompt Tuning](https://mlsanity.com/paper/2303.12214/) (2023)
- [MedPrompt: Cross-Modal Prompting for Multi-Task Medical Image Translation](https://mlsanity.com/paper/2310.02663/) (2023)
- [Towards a text-based quantitative and explainable histopathology image analysis](https://mlsanity.com/paper/2407.07360/) (2024)

## Built on by (most cited first, 30 of 35)

- [Multimodal Model for Computational Pathology:Representation Learning and Image Compression](https://mlsanity.com/paper/2603.18660/) (2026)
- [Act Like a Pathologist: Tissue-Aware Whole Slide Image Reasoning](https://mlsanity.com/paper/2603.00667/) (2026)
- [Small Vision-Language Models: A Survey on Compact Architectures and Techniques](https://mlsanity.com/paper/2503.10665/) (2025)
- [Mamba2D: A Natively Multi-Dimensional State-Space Model for Vision Tasks](https://mlsanity.com/paper/2412.16146/) (2024)
- [GeoMaNO: Geometric Mamba Neural Operator for Partial Differential Equations](https://mlsanity.com/paper/2505.12020/) (2025)
- [pLSTM: parallelizable Linear Source Transition Mark networks](https://mlsanity.com/paper/2506.11997/) (2025)
- [SGPMIL: Sparse Gaussian Process Multiple Instance Learning](https://mlsanity.com/paper/2507.08711/) (2025)
- [ME-Mamba: Multi-Expert Mamba with Efficient Knowledge Capture and Fusion for Multimodal Survival Analysis](https://mlsanity.com/paper/2509.16900/) (2025)
- [Towards Ancient Plant Seed Classification: A Benchmark Dataset and Baseline Model](https://mlsanity.com/paper/2512.18247/) (2025)
- [Cell-Type Prototype-Informed Neural Network for Gene Expression Estimation from Pathology Images](https://mlsanity.com/paper/2603.18461/) (2026)
- [VLEER: Vision and Language Embeddings for Explainable Whole Slide Image Representation](https://mlsanity.com/paper/2502.20850/) (2025)
- [Polyline Path Masked Attention for Vision Transformer](https://mlsanity.com/paper/2506.15940/) (2025)
- [LBMamba: Locally Bi-directional Mamba](https://mlsanity.com/paper/2506.15976/) (2025)
- [PathMamba: A Hybrid Mamba-Transformer for Topologically Coherent Road Segmentation in Satellite Imagery](https://mlsanity.com/paper/2511.21298/) (2025)
- [MambaMIL+: Modeling Long-Term Contextual Patterns for Gigapixel Whole Slide Image](https://mlsanity.com/paper/2512.17726/) (2025)
- [CellMamba: Adaptive Mamba for Accurate and Efficient Cell Detection](https://mlsanity.com/paper/2512.21803/) (2025)
- [OCTOPUS: Enhancing the Spatial-Awareness of Vision SSMs with Multi-Dimensional Scans and Traversal Selection](https://mlsanity.com/paper/2602.00904/) (2026)
- [Partial Ring Scan: Revisiting Scan Order in Vision State Space Models](https://mlsanity.com/paper/2602.04170/) (2026)
- [Revisiting Data Challenges of Computational Pathology: A Pack-based Multiple Instance Learning Training Framework](https://mlsanity.com/paper/2509.20923/) (2025)
- [Gather-Scatter Mamba: Accelerating Propagation with Efficient State Space Model](https://mlsanity.com/paper/2510.00862/) (2025)
- [DeltaMIL: Gated Memory Integration for Efficient and Discriminative Whole Slide Image Analysis](https://mlsanity.com/paper/2512.19331/) (2025)
- [Learning to Sense for Driving: Joint Optics-Sensor-Model Co-Design for Semantic Segmentation](https://mlsanity.com/paper/2512.20815/) (2025)
- [A Multi-scale Linear-time Encoder for Whole-Slide Image Analysis](https://mlsanity.com/paper/2602.02918/) (2026)
- [AINet: Anchor Instances Learning for Regional Heterogeneity in Whole Slide Image](https://mlsanity.com/paper/2602.20187/) (2026)
- [Permutation-Equivariant 2D State Space Models: Theory and Canonical Architecture for Multivariate Time Series](https://mlsanity.com/paper/2603.08753/) (2026)
- [Do VLMs Need Vision Transformers? Evaluating State Space Models as Vision Encoders](https://mlsanity.com/paper/2603.19209/) (2026)
- [Structured State-Space Regularization for Generation-Friendly Image Tokenization](https://mlsanity.com/paper/2604.11089/) (2026)
- [TCP-SSM: Efficient Vision State Space Models with Token-Conditioned Poles](https://mlsanity.com/paper/2605.11563/) (2026)
- [Spatial Blindness in Whole-Slide Multiple Instance Learning](https://mlsanity.com/paper/2605.17449/) (2026)
- [MambaRaw: Selective State Space Modeling for Efficient 4K Raw Image Reconstruction](https://mlsanity.com/paper/2606.24479/) (2026)

## Abstract (arXiv)

Efficiently modeling large 2D contexts is essential for various fields including Giga-Pixel Whole Slide Imaging (WSI) and remote sensing. Transformer-based models offer high parallelism but face challenges due to their quadratic complexity for handling long sequences. Recently, Mamba introduced a selective State Space Model (SSM) with linear complexity and high parallelism, enabling effective and efficient modeling of wide context in 1D sequences. However, extending Mamba to vision tasks, which inherently involve 2D structures, results in spatial discrepancies due to the limitations of 1D sequence processing. On the other hand, current 2D SSMs inherently model 2D structures but they suffer from prohibitively slow computation due to the lack of efficient parallel algorithms. In this work, we propose 2DMamba, a novel 2D selective SSM framework that incorporates the 2D spatial structure of images into Mamba, with a highly optimized hardware-aware operator, adopting both spatial continuity and computational efficiency. We validate the versatility of our approach on both WSIs and natural images. Extensive experiments on 10 public datasets for WSI classification and survival analysis show that 2DMamba improves up to 2.48% in AUC, 3.11% in F1 score, 2.47% in accuracy and 5.52% in C-index. Additionally, integrating our method with VMamba for natural imaging yields 0.5 to 0.7 improvements in mIoU on the ADE20k semantic segmentation dataset, and 0.2% accuracy improvement on ImageNet-1K classification dataset. Our code is available at https://github.com/AtlasAnalyticsLab/2DMamba.