# UrbanSAM: Learning Invariance-Inspired Adapters for Segment Anything Models in Urban Construction

arXiv: https://arxiv.org/abs/2502.15199 · HTML: https://mlsanity.com/paper/2502.15199/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 51 references that are in the map.
- First posted on arXiv 21 Feb 2025.

## Builds on (most cited first, 24 of 24)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [Release Strategies and the Social Impacts of Language Models](https://mlsanity.com/paper/1908.09203/) (2019)
- [Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation](https://mlsanity.com/paper/2304.12620/) (2023)
- [Customized Segment Anything Model for Medical Image Segmentation](https://mlsanity.com/paper/2304.13785/) (2023)
- [SpectralGPT: Spectral Remote Sensing Foundation Model](https://mlsanity.com/paper/2311.07113/) (2023)
- [RSPrompter: Learning to Prompt for Remote Sensing Instance Segmentation based on Visual Foundation Model](https://mlsanity.com/paper/2306.16269/) (2023)
- [The Segment Anything Model (SAM) for Remote Sensing Applications: From Zero to One Shot](https://mlsanity.com/paper/2306.16623/) (2023)
- [LooseControl: Lifting ControlNet for Generalized Depth Conditioning](https://mlsanity.com/paper/2312.03079/) (2023)
- [Segment Any Anomaly without Training via Hybrid Prompt Regularization](https://mlsanity.com/paper/2305.10724/) (2023)
- [Cross-City Matters: A Multimodal Remote Sensing Benchmark Dataset for Cross-City Semantic Segmentation using High-Resolution Domain Adaptation Networks](https://mlsanity.com/paper/2309.16499/) (2023)
- [ClipSAM: CLIP and SAM Collaboration for Zero-Shot Anomaly Segmentation](https://mlsanity.com/paper/2401.12665/) (2024)
- [UV-SAM: Adapting Segment Anything Model for Urban Village Identification](https://mlsanity.com/paper/2401.08083/) (2024)
- [Segment Anything Model for Road Network Graph Extraction](https://mlsanity.com/paper/2403.16051/) (2024)
- [GPT-4V-AD: Exploring Grounding Potential of VQA-oriented GPT-4V for Zero-shot Anomaly Detection](https://mlsanity.com/paper/2311.02612/) (2023)
- [GLH-Water: A Large-Scale Dataset for Global Surface Water Detection in Large-Size Very-High-Resolution Satellite Imagery](https://mlsanity.com/paper/2303.09310/) (2023)
- [Calib-Anything: Zero-training LiDAR-Camera Extrinsic Calibration Method Using Segment Anything](https://mlsanity.com/paper/2306.02656/) (2023)
- [HiFi Tuner: High-Fidelity Subject-Driven Fine-Tuning for Diffusion Models](https://mlsanity.com/paper/2312.00079/) (2023)
- [RSAM-Seg: A SAM-based Approach with Prior Knowledge Integration for Remote Sensing Image Semantic Segmentation](https://mlsanity.com/paper/2402.19004/) (2024)
- [Event-Free Moving Object Segmentation from Moving Ego Vehicle](https://mlsanity.com/paper/2305.00126/) (2023)
- [SolarSAM: Building-scale Photovoltaic Potential Assessment Based on Segment Anything Model (SAM) and Remote Sensing for Emerging City](https://mlsanity.com/paper/2407.00296/) (2024)

## Built on by (most cited first, 8 of 8)

- [FlexiMo: A Flexible Remote Sensing Foundation Model](https://mlsanity.com/paper/2503.23844/) (2025)
- [Foundation Models in Remote Sensing: Evolving from Unimodality to Multimodality](https://mlsanity.com/paper/2603.00988/) (2026)
- [CLDTracker: A Comprehensive Language Description for Visual Tracking](https://mlsanity.com/paper/2505.23704/) (2025)
- [Hyperspectral Imaging](https://mlsanity.com/paper/2508.08107/) (2025)
- [Multimodal Feature Fusion Network with Text Difference Enhancement for Remote Sensing Change Detection](https://mlsanity.com/paper/2509.03961/) (2025)
- [AMD-HookNet++: Evolution of AMD-HookNet with Hybrid CNN-Transformer Feature Enhancement for Glacier Calving Front Segmentation](https://mlsanity.com/paper/2512.14639/) (2025)
- [Hyper-spectral Unmixing algorithms for remote compositional surface mapping: a review of the state of the art](https://mlsanity.com/paper/2507.14260/) (2025)
- [LandSegmenter: Towards a Flexible Foundation Model for Land Use and Land Cover Mapping](https://mlsanity.com/paper/2511.08156/) (2025)

## Abstract (arXiv)

Object extraction and segmentation from remote sensing (RS) images is a critical yet challenging task in urban environment monitoring. Urban morphology is inherently complex, with irregular objects of diverse shapes and varying scales. These challenges are amplified by heterogeneity and scale disparities across RS data sources, including sensors, platforms, and modalities, making accurate object segmentation particularly demanding. While the Segment Anything Model (SAM) has shown significant potential in segmenting complex scenes, its performance in handling form-varying objects remains limited due to manual-interactive prompting. To this end, we propose UrbanSAM, a customized version of SAM specifically designed to analyze complex urban environments while tackling scaling effects from remotely sensed observations. Inspired by multi-resolution analysis (MRA) theory, UrbanSAM incorporates a novel learnable prompter equipped with a Uscaling-Adapter that adheres to the invariance criterion, enabling the model to capture multiscale contextual information of objects and adapt to arbitrary scale variations with theoretical guarantees. Furthermore, features from the Uscaling-Adapter and the trunk encoder are aligned through a masked cross-attention operation, allowing the trunk encoder to inherit the adapter's multiscale aggregation capability. This synergy enhances the segmentation performance, resulting in more powerful and accurate outputs, supported by the learned adapter. Extensive experimental results demonstrate the flexibility and superior segmentation performance of the proposed UrbanSAM on a global-scale dataset, encompassing scale-varying urban objects such as buildings, roads, and water.