# RSBuilding: Towards General Remote Sensing Image Building Extraction and Change Detection with Foundation Model

arXiv: https://arxiv.org/abs/2403.07564 · HTML: https://mlsanity.com/paper/2403.07564/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 92 references that are in the map.
- First posted on arXiv 12 Mar 2024.

## Builds on (most cited first, 30 of 37)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers](https://mlsanity.com/paper/2105.15203/) (2021)
- [MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models](https://mlsanity.com/paper/2304.10592/) (2023)
- [Rethinking Atrous Convolution for Semantic Image Segmentation](https://mlsanity.com/paper/1706.05587/) (2017)
- [Fully Convolutional Siamese Networks for Change Detection](https://mlsanity.com/paper/1810.08462/) (2018)
- [Remote Sensing Image Change Detection with Transformers](https://mlsanity.com/paper/2103.00208/) (2021)
- [A Transformer-Based Siamese Network for Change Detection](https://mlsanity.com/paper/2201.01293/) (2022)
- [RSPrompter: Learning to Prompt for Remote Sensing Instance Segmentation based on Visual Foundation Model](https://mlsanity.com/paper/2306.16269/) (2023)
- [In-Context Impersonation Reveals Large Language Models' Strengths and Biases](https://mlsanity.com/paper/2305.14930/) (2023)
- [Changer: Feature Interaction is What You Need for Change Detection](https://mlsanity.com/paper/2209.08290/) (2022)
- [S2Looking: A Satellite Side-Looking Dataset for Building Change Detection](https://mlsanity.com/paper/2107.09244/) (2021)
- [Transformers in Remote Sensing: A Survey](https://mlsanity.com/paper/2209.01206/) (2022)
- [A New Learning Paradigm for Foundation Model-based Remote Sensing Change Detection](https://mlsanity.com/paper/2312.01163/) (2023)
- [OvarNet: Towards Open-vocabulary Object Attribute Recognition](https://mlsanity.com/paper/2301.09506/) (2023)
- [TINYCD: A (Not So) Deep Learning Model For Change Detection](https://mlsanity.com/paper/2207.13159/) (2022)
- [Semantic Change Detection with Asymmetric Siamese Networks](https://mlsanity.com/paper/2010.05687/) (2020)

## Built on by (most cited first, 12 of 12)

- [FlexiMo: A Flexible Remote Sensing Foundation Model](https://mlsanity.com/paper/2503.23844/) (2025)
- [Semantic-CC: Boosting Remote Sensing Image Change Captioning via Foundational Knowledge and Semantic Guidance](https://mlsanity.com/paper/2407.14032/) (2024)
- [UniChange: Unifying Change Detection with Multimodal Large Language Model](https://mlsanity.com/paper/2511.02607/) (2025)
- [Exploring Foundation Models in Remote Sensing Image Change Detection: A Comprehensive Survey](https://mlsanity.com/paper/2410.07824/) (2024)
- [MarsSeg: Mars Surface Semantic Segmentation with Multi-level Extractor and Connector](https://mlsanity.com/paper/2404.04155/) (2024)
- [RSEdit: Text-Guided Image Editing for Remote Sensing](https://mlsanity.com/paper/2603.13708/) (2026)
- [A Remote Sensing Image Change Detection Method Integrating Layer Exchange and Channel-Spatial Differences](https://mlsanity.com/paper/2501.10905/) (2025)
- [MSSFC-Net:Enhancing Building Interpretation with Multi-Scale Spatial-Spectral Feature Collaboration](https://mlsanity.com/paper/2504.00759/) (2025)
- [LTCA: Long-range Temporal Context Attention for Referring Video Object Segmentation](https://mlsanity.com/paper/2510.08305/) (2025)
- [Exchange Is All You Need for Remote Sensing Change Detection](https://mlsanity.com/paper/2601.07805/) (2026)
- [Building Extraction from Remote Sensing Imagery under Hazy and Low-light Conditions: Benchmark and Baseline](https://mlsanity.com/paper/2604.15088/) (2026)
- [Developing a foundation model for high-resolution remote sensing data of the Netherlands](https://mlsanity.com/paper/2605.10184/) (2026)

## Abstract (arXiv)

The intelligent interpretation of buildings plays a significant role in urban planning and management, macroeconomic analysis, population dynamics, etc. Remote sensing image building interpretation primarily encompasses building extraction and change detection. However, current methodologies often treat these two tasks as separate entities, thereby failing to leverage shared knowledge. Moreover, the complexity and diversity of remote sensing image scenes pose additional challenges, as most algorithms are designed to model individual small datasets, thus lacking cross-scene generalization. In this paper, we propose a comprehensive remote sensing image building understanding model, termed RSBuilding, developed from the perspective of the foundation model. RSBuilding is designed to enhance cross-scene generalization and task universality. Specifically, we extract image features based on the prior knowledge of the foundation model and devise a multi-level feature sampler to augment scale information. To unify task representation and integrate image spatiotemporal clues, we introduce a cross-attention decoder with task prompts. Addressing the current shortage of datasets that incorporate annotations for both tasks, we have developed a federated training strategy to facilitate smooth model convergence even when supervision for some tasks is missing, thereby bolstering the complementarity of different tasks. Our model was trained on a dataset comprising up to 245,000 images and validated on multiple building extraction and change detection datasets. The experimental results substantiate that RSBuilding can concurrently handle two structurally distinct tasks and exhibits robust zero-shot generalization capabilities.