# BuildMamba: A Visual State-Space Based Model for Multi-Task Building Segmentation and Height Estimation from Satellite Images

arXiv: https://arxiv.org/abs/2603.08523 · HTML: https://mlsanity.com/paper/2603.08523/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in Point clouds, detection and remote sensing, one of 10 directions in State-space sequence models (S4, Mamba).
- Builds on 22 of its 44 references that are in the map.
- First posted on arXiv 9 Mar 2026.

## Placed in

- State-space sequence models (S4, Mamba) / [Point clouds, detection and remote sensing](https://mlsanity.com/ssm/point-clouds-detection-and-remote-sensing/)

## Builds on (most cited first, 22 of 22)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [VMamba: Visual State Space Model](https://mlsanity.com/paper/2401.10166/) (2024)
- [Deep Ordinal Regression Network for Monocular Depth Estimation](https://mlsanity.com/paper/1806.02446/) (2018)
- [AdaBins: Depth Estimation using Adaptive Bins](https://mlsanity.com/paper/2011.14141/) (2020)
- [BinsFormer: Revisiting Adaptive Bins for Monocular Depth Estimation](https://mlsanity.com/paper/2204.00987/) (2022)
- [Foreground-Aware Relation Network for Geospatial Object Segmentation in High Spatial Resolution Remote Sensing Imagery](https://mlsanity.com/paper/2011.09766/) (2020)
- [Semantic Stereo for Incidental Satellite Images](https://mlsanity.com/paper/1811.08739/) (2018)
- [Continental-Scale Building Detection from High Resolution Satellite Imagery](https://mlsanity.com/paper/2107.12283/) (2021)
- [IM2HEIGHT: Height Estimation from Single Monocular Imagery via Fully Residual Convolutional-Deconvolutional Network](https://mlsanity.com/paper/1802.10249/) (2018)
- [PointFlow: Flowing Semantics Through Points for Aerial Image Segmentation](https://mlsanity.com/paper/2103.06564/) (2021)
- [Building extraction with vision transformer](https://mlsanity.com/paper/2111.15637/) (2021)
- [Building Extraction from Remote Sensing Images via an Uncertainty-Aware Network](https://mlsanity.com/paper/2307.12309/) (2023)
- [HTC-DC Net: Monocular Height Estimation from Single Remote Sensing Images](https://mlsanity.com/paper/2309.16486/) (2023)
- [LIGHT: Joint Individual Building Extraction and Height Estimation from Satellite Images through a Unified Multitask Learning Network](https://mlsanity.com/paper/2304.01090/) (2023)
- [Aerial Height Prediction and Refinement Neural Networks with Semantic and Geometric Guidance](https://mlsanity.com/paper/2011.10697/) (2020)
- [Data Fusion for Multi-Task Learning of Building Extraction and Height Estimation](https://mlsanity.com/paper/2308.02960/) (2023)
- [TransAdapter: Vision Transformer for Feature-Centric Unsupervised Domain Adaptation](https://mlsanity.com/paper/2412.04073/) (2024)
- [HGDNet: A Height-Hierarchy Guided Dual-Decoder Network for Single View Building Extraction and Height Estimation](https://mlsanity.com/paper/2308.05387/) (2023)

## Built on by (most cited first, 1 of 1)

- [Joint Instance Segmentation and Geometric Attribute Regression for Roof Structures in Aerial Imagery](https://mlsanity.com/paper/2605.26370/) (2026)

## Abstract (arXiv)

Accurate building segmentation and height estimation from single-view RGB satellite imagery are fundamental for urban analytics, yet remain ill-posed due to structural variability and the high computational cost of global context modeling. While current approaches typically adapt monocular depth architectures, they often suffer from boundary bleeding and systematic underestimation of high-rise structures. To address these limitations, we propose BuildMamba, a unified multi-task framework designed to exploit the linear-time global modeling of visual state-space models. Motivated by the need for stronger structural coupling and computational efficiency, we introduce three modules: a Mamba Attention Module for dynamic spatial recalibration, a Spatial-Aware Mamba-FPN for multi-scale feature aggregation via gated state-space scans, and a Mask-Aware Height Refinement module using semantic priors to suppress height artifacts. Extensive experiments demonstrate that BuildMamba establishes a new performance upper bound across three benchmarks. Specifically, it achieves an IoU of 0.93 and RMSE of 1.77~m on DFC23 benchmark, surpassing state-of-the-art by 0.82~m in height estimation. Simulation results confirm the model's superior robustness and scalability for large-scale 3D urban reconstruction.