# SAM-Assisted Remote Sensing Imagery Semantic Segmentation with Object and Boundary Constraints

arXiv: https://arxiv.org/abs/2312.02464 · HTML: https://mlsanity.com/paper/2312.02464/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 81 references that are in the map.
- First posted on arXiv 5 Dec 2023.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Fast Segment Anything](https://mlsanity.com/paper/2306.12156/) (2023)
- [ResUNet-a: a deep learning framework for semantic segmentation of remotely sensed data](https://mlsanity.com/paper/1904.00592/) (2019)
- [LoveDA: A Remote Sensing Land-Cover Dataset for Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2110.08733/) (2021)
- [SpectralGPT: Spectral Remote Sensing Foundation Model](https://mlsanity.com/paper/2311.07113/) (2023)
- [Deep Convolutional Networks on Graph-Structured Data](https://mlsanity.com/paper/1506.05163/) (2015)
- [RSPrompter: Learning to Prompt for Remote Sensing Instance Segmentation based on Visual Foundation Model](https://mlsanity.com/paper/2306.16269/) (2023)
- [UNetFormer: A UNet-like Transformer for Efficient Semantic Segmentation of Remote Sensing Urban Scene Imagery](https://mlsanity.com/paper/2109.08937/) (2021)
- [The Segment Anything Model (SAM) for Remote Sensing Applications: From Zero to One Shot](https://mlsanity.com/paper/2306.16623/) (2023)
- [Foreground-Aware Relation Network for Geospatial Object Segmentation in High Spatial Resolution Remote Sensing Imagery](https://mlsanity.com/paper/2011.09766/) (2020)
- [Beyond RGB: Very High Resolution Urban Remote Sensing With Multimodal Deep Networks](https://mlsanity.com/paper/1711.08681/) (2017)
- [Multimodal Fusion Transformer for Remote Sensing Image Classification](https://mlsanity.com/paper/2203.16952/) (2022)
- [A Novel Transformer Based Semantic Segmentation Scheme for Fine-Resolution Remote Sensing Images](https://mlsanity.com/paper/2104.12137/) (2021)
- [Input Augmentation with SAM: Boosting Medical Image Segmentation with Segmentation Foundation Model](https://mlsanity.com/paper/2304.11332/) (2023)
- [Adapting Segment Anything Model for Change Detection in HR Remote Sensing Images](https://mlsanity.com/paper/2309.01429/) (2023)
- [Cross-City Matters: A Multimodal Remote Sensing Benchmark Dataset for Cross-City Semantic Segmentation using High-Resolution Domain Adaptation Networks](https://mlsanity.com/paper/2309.16499/) (2023)
- [Text2Seg: Remote Sensing Image Semantic Segmentation via Text-Guided Visual Foundation Models](https://mlsanity.com/paper/2304.10597/) (2023)
- [Segment anything, from space?](https://mlsanity.com/paper/2304.13000/) (2023)
- [Classification With an Edge: Improving Semantic Image Segmentation with Boundary Detection](https://mlsanity.com/paper/1612.01337/) (2016)
- [SAMRS: Scaling-up Remote Sensing Segmentation Dataset with Segment Anything Model](https://mlsanity.com/paper/2305.02034/) (2023)
- [Boundary Loss for Remote Sensing Imagery Semantic Segmentation](https://mlsanity.com/paper/1905.07852/) (2019)
- [Segment Anything is A Good Pseudo-label Generator for Weakly Supervised Semantic Segmentation](https://mlsanity.com/paper/2305.01275/) (2023)
- [Knowledge distillation with Segment Anything (SAM) model for Planetary Geological Mapping](https://mlsanity.com/paper/2305.07586/) (2023)
- [nnSAM: Plug-and-play Segment Anything Model Improves nnUNet Performance](https://mlsanity.com/paper/2309.16967/) (2023)
- [Land Cover Classification from Remote Sensing Images Based on Multi-Scale Fully Convolutional Network](https://mlsanity.com/paper/2008.00168/) (2020)

## Built on by (most cited first, 30 of 35)

- [A Unified Framework with Multimodal Fine-tuning for Remote Sensing Semantic Segmentation](https://mlsanity.com/paper/2410.11160/) (2024)
- [ROS-SAM: High-Quality Interactive Segmentation for Remote Sensing Moving Object](https://mlsanity.com/paper/2503.12006/) (2025)
- [Decomposition-based Unsupervised Domain Adaptation for Remote Sensing Image Semantic Segmentation](https://mlsanity.com/paper/2404.04531/) (2024)
- [Unveiling the Potential of Segment Anything Model 2 for RGB-Thermal Semantic Segmentation with Language Guidance](https://mlsanity.com/paper/2503.02581/) (2025)
- [No time to train! Training-Free Reference-Based Instance Segmentation](https://mlsanity.com/paper/2507.02798/) (2025)
- [A Multispectral Automated Transfer Technique (MATT) for machine-driven image labeling utilizing the Segment Anything Model (SAM)](https://mlsanity.com/paper/2402.11413/) (2024)
- [MMM-RS: A Multi-modal, Multi-GSD, Multi-scene Remote Sensing Dataset and Benchmark for Text-to-Image Generation](https://mlsanity.com/paper/2410.22362/) (2024)
- [PPMamba: A Pyramid Pooling Local Auxiliary SSM-Based Model for Remote Sensing Image Semantic Segmentation](https://mlsanity.com/paper/2409.06309/) (2024)
- [Auto-Prompting SAM for Weakly Supervised Landslide Extraction](https://mlsanity.com/paper/2501.13426/) (2025)
- [DF4LCZ: A SAM-Empowered Data Fusion Framework for Scene-Level Local Climate Zone Classification](https://mlsanity.com/paper/2403.09367/) (2024)
- [Efficient Redundancy Reduction for Open-Vocabulary Semantic Segmentation](https://mlsanity.com/paper/2501.17642/) (2025)
- [Expose Camouflage in the Water: Underwater Camouflaged Instance Segmentation and Dataset](https://mlsanity.com/paper/2510.17585/) (2025)
- [Slender Object Scene Segmentation in Remote Sensing Image Based on Learnable Morphological Skeleton with Segment Anything Model](https://mlsanity.com/paper/2411.08592/) (2024)
- [Kolmogorov-Arnold Network for Remote Sensing Image Semantic Segmentation](https://mlsanity.com/paper/2501.07390/) (2025)
- [SAM2-ELNet: Label Enhancement and Automatic Annotation for Remote Sensing Segmentation](https://mlsanity.com/paper/2503.12404/) (2025)
- [ClassWise-CRF: Category-Specific Fusion for Enhanced Semantic Segmentation of Remote Sensing Imagery](https://mlsanity.com/paper/2504.21491/) (2025)
- [WakeupUrban: Unsupervised Semantic Segmentation of Mid-20th century Urban Landscapes with Satellite Imagery](https://mlsanity.com/paper/2506.09476/) (2025)
- [ViRefSAM: Visual Reference-Guided Segment Anything Model for Remote Sensing Segmentation](https://mlsanity.com/paper/2507.02294/) (2025)
- [Weakly-supervised Contrastive Learning with Quantity Prompts for Moving Infrared Small Target Detection](https://mlsanity.com/paper/2507.02454/) (2025)
- [ReSAM: Refine, Requery, and Reinforce: Self-Prompting Point-Supervised Segmentation for Remote Sensing Images](https://mlsanity.com/paper/2511.21606/) (2025)
- [RAH-VLA: Resolution-Adaptive Hierarchical Vision-Language Alignment for Multimodal Remote Sensing Understanding](https://mlsanity.com/paper/2512.23243/) (2025)
- [Structure-Semantic Decoupled Modulation of Global Geospatial Embeddings for High-Resolution Remote Sensing Mapping](https://mlsanity.com/paper/2604.19591/) (2026)
- [Remote SAMsing: From Segment Anything to Segment Everything](https://mlsanity.com/paper/2605.00256/) (2026)
- [SinkSAM-Net: Knowledge-Driven Self-Supervised Sinkhole Segmentation Using Topographic Priors and Segment Anything Model](https://mlsanity.com/paper/2410.01473/) (2024)
- [PolSAM: Polarimetric Scattering Mechanism Informed Segment Anything Model](https://mlsanity.com/paper/2412.12737/) (2024)
- [Breaking the Box: Enhancing Remote Sensing Image Segmentation with Freehand Sketches](https://mlsanity.com/paper/2503.12191/) (2025)
- [Exploring Generalizable Pre-training for Real-world Change Detection via Geometric Estimation](https://mlsanity.com/paper/2504.14306/) (2025)
- [NSegment : Label-specific Deformations for Remote Sensing Image Segmentation](https://mlsanity.com/paper/2504.19634/) (2025)
- [Baltimore Atlas: FreqWeaver Adapter for Semi-supervised Ultra-high Spatial Resolution Land Cover Classification](https://mlsanity.com/paper/2506.15565/) (2025)
- [AMMNet: An Asymmetric Multi-Modal Network for Remote Sensing Semantic Segmentation](https://mlsanity.com/paper/2507.16158/) (2025)

## Abstract (arXiv)

Semantic segmentation of remote sensing imagery plays a pivotal role in extracting precise information for diverse down-stream applications. Recent development of the Segment Anything Model (SAM), an advanced general-purpose segmentation model, has revolutionized this field, presenting new avenues for accurate and efficient segmentation. However, SAM is limited to generating segmentation results without class information. Consequently, the utilization of such a powerful general vision model for semantic segmentation in remote sensing images has become a focal point of research. In this paper, we present a streamlined framework aimed at leveraging the raw output of SAM by exploiting two novel concepts called SAM-Generated Object (SGO) and SAM-Generated Boundary (SGB). More specifically, we propose a novel object loss and further introduce a boundary loss as augmentative components to aid in model optimization in a general semantic segmentation framework. Taking into account the content characteristics of SGO, we introduce the concept of object consistency to leverage segmented regions lacking semantic information. By imposing constraints on the consistency of predicted values within objects, the object loss aims to enhance semantic segmentation performance. Furthermore, the boundary loss capitalizes on the distinctive features of SGB by directing the model's attention to the boundary information of the object. Experimental results on two well-known datasets, namely ISPRS Vaihingen and LoveDA Urban, demonstrate the effectiveness of our proposed method. The source code for this work will be accessible at https://github.com/sstary/SSRS.