# Towards Open-Vocabulary Remote Sensing Image Semantic Segmentation

arXiv: https://arxiv.org/abs/2412.19492 · HTML: https://mlsanity.com/paper/2412.19492/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 46 references that are in the map.
- First posted on arXiv 27 Dec 2024.

## Builds on (most cited first, 30 of 30)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [COCO-Stuff: Thing and Stuff Classes in Context](https://mlsanity.com/paper/1612.03716/) (2016)
- [DOTA: A Large-scale Dataset for Object Detection in Aerial Images](https://mlsanity.com/paper/1711.10398/) (2017)
- [Open-Vocabulary Semantic Segmentation with Mask-adapted CLIP](https://mlsanity.com/paper/2210.04150/) (2022)
- [Scaling Open-Vocabulary Image Segmentation with Image-Level Labels](https://mlsanity.com/paper/2112.12143/) (2021)
- [RemoteCLIP: A Vision Language Foundation Model for Remote Sensing](https://mlsanity.com/paper/2306.11029/) (2023)
- [Object Detection in Optical Remote Sensing Images: A Survey and A New Benchmark](https://mlsanity.com/paper/1909.00133/) (2019)
- [Side Adapter Network for Open-Vocabulary Semantic Segmentation](https://mlsanity.com/paper/2302.12242/) (2023)
- [DeepGlobe 2018: A Challenge to Parse the Earth through Satellite Images](https://mlsanity.com/paper/1805.06561/) (2018)
- [LoveDA: A Remote Sensing Land-Cover Dataset for Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2110.08733/) (2021)
- [A Simple Baseline for Open-Vocabulary Semantic Segmentation with Pre-trained Vision-language Model](https://mlsanity.com/paper/2112.14757/) (2021)
- [CAT-Seg: Cost Aggregation for Open-Vocabulary Semantic Segmentation](https://mlsanity.com/paper/2303.11797/) (2023)
- [iSAID: A Large-scale Dataset for Instance Segmentation in Aerial Images](https://mlsanity.com/paper/1905.12886/) (2019)
- [FloodNet: A High Resolution Aerial Imagery Dataset for Post Flood Scene Understanding](https://mlsanity.com/paper/2012.02951/) (2020)
- [SpaceNet: A Remote Sensing Dataset and Challenge Series](https://mlsanity.com/paper/1807.01232/) (2018)
- [SED: A Simple Encoder-Decoder for Open-Vocabulary Semantic Segmentation](https://mlsanity.com/paper/2311.15537/) (2023)
- [Foreground-Aware Relation Network for Geospatial Object Segmentation in High Spatial Resolution Remote Sensing Imagery](https://mlsanity.com/paper/2011.09766/) (2020)
- [Open-Vocabulary Segmentation with Semantic-Assisted Calibration](https://mlsanity.com/paper/2312.04089/) (2023)
- [FLAIR: a Country-Scale Land Cover Semantic Segmentation Dataset From Multi-Source Optical Imagery](https://mlsanity.com/paper/2310.13336/) (2023)
- [Open-Vocabulary Semantic Segmentation with Image Embedding Balancing](https://mlsanity.com/paper/2406.09829/) (2024)
- [Semantic Segmentation of Remote Sensing Images with Sparse Annotations](https://mlsanity.com/paper/2101.03492/) (2021)
- [AerialFormer: Multi-resolution Transformer for Aerial Image Segmentation](https://mlsanity.com/paper/2306.06842/) (2023)
- [Fully Transformer Network for Change Detection of Remote Sensing Images](https://mlsanity.com/paper/2210.00757/) (2022)
- [Unsupervised domain adaptation semantic segmentation of high-resolution remote sensing imagery with invariant domain-level prototype memory](https://mlsanity.com/paper/2208.07722/) (2022)
- [Large-scale Land Cover Classification in GaoFen-2 Satellite Imagery](https://mlsanity.com/paper/1806.00901/) (2018)

## Built on by (most cited first, 19 of 19)

- [Exploring Efficient Open-Vocabulary Segmentation in the Remote Sensing](https://mlsanity.com/paper/2509.12040/) (2025)
- [SegEarth-OV3: Exploring SAM 3 for Open-Vocabulary Semantic Segmentation in Remote Sensing Images](https://mlsanity.com/paper/2512.08730/) (2025)
- [EarthSynth: Generating Informative Earth Observation with Diffusion Models](https://mlsanity.com/paper/2505.12108/) (2025)
- [Video-Level Language-Driven Video-Based Visible-Infrared Person Re-Identification](https://mlsanity.com/paper/2506.02439/) (2025)
- [InstructSAM: A Training-Free Framework for Instruction-Oriented Remote Sensing Object Recognition](https://mlsanity.com/paper/2505.15818/) (2025)
- [SCORE: Scene Context Matters in Open-Vocabulary Remote Sensing Instance Segmentation](https://mlsanity.com/paper/2507.12857/) (2025)
- [Annotation-Free Open-Vocabulary Segmentation for Remote-Sensing Images](https://mlsanity.com/paper/2508.18067/) (2025)
- [Generalizable Multispectral Land Cover Classification via Frequency-Aware Mixture of Low-Rank Token Experts](https://mlsanity.com/paper/2505.14088/) (2025)
- [OmniEarth: A Benchmark for Evaluating Vision-Language Models in Geospatial Tasks](https://mlsanity.com/paper/2603.09471/) (2026)
- [AerialMetric: Benchmarking and Adapting UAV Monocular Metric Depth Estimation in the Real World](https://mlsanity.com/paper/2606.29716/) (2026)
- [Toward Stable Semi-Supervised Remote Sensing Segmentation via Co-Guidance and Co-Fusion](https://mlsanity.com/paper/2512.23035/) (2025)
- [Vision-Language Model Purified Semi-Supervised Semantic Segmentation for Remote Sensing Images](https://mlsanity.com/paper/2602.00202/) (2026)
- [Standalone DINOv3 for Training-Free Open-Vocabulary Semantic Segmentation in Remote Sensing](https://mlsanity.com/paper/2608.03023/) (2026)
- [Geospatial-Reasoning-Driven Vocabulary-Agnostic Remote Sensing Semantic Segmentation](https://mlsanity.com/paper/2602.08206/) (2026)
- [Open-Vocabulary Semantic Segmentation in Remote Sensing via Hierarchical Attention Masking and Model Composition](https://mlsanity.com/paper/2602.23869/) (2026)
- [Training-Free Open-Vocabulary Visual Grounding for Remote Sensing Images and Videos](https://mlsanity.com/paper/2606.16124/) (2026)
- [RSGPNet: Geometric Prompting for Remote Sensing Open-Vocabulary Semantic Segmentation](https://mlsanity.com/paper/2606.28410/) (2026)
- [Frequency and Edge-Guided Segment Anything Model for Remote Sensing Image Semantic Segmentation](https://mlsanity.com/paper/2608.15054/) (2026)
- [A Geometric-Transformation Feature-Adaptive Manifold Restoration Method for Open-Vocabulary Semantic Segmentation of Remote Sensing Images](https://mlsanity.com/paper/2610.04300/) (2026)

## Abstract (arXiv)

Recently, deep learning based methods have revolutionized remote sensing image segmentation. However, these methods usually rely on a pre-defined semantic class set, thus needing additional image annotation and model training when adapting to new classes. More importantly, they are unable to segment arbitrary semantic classes. In this work, we introduce Open-Vocabulary Remote Sensing Image Semantic Segmentation (OVRSISS), which aims to segment arbitrary semantic classes in remote sensing images. To address the lack of OVRSISS datasets, we develop LandDiscover50K, a comprehensive dataset of 51,846 images covering 40 diverse semantic classes. In addition, we propose a novel framework named GSNet that integrates domain priors from special remote sensing models and versatile capabilities of general vision-language models. Technically, GSNet consists of a Dual-Stream Image Encoder (DSIE), a Query-Guided Feature Fusion (QGFF), and a Residual Information Preservation Decoder (RIPD). DSIE first captures comprehensive features from both special models and general models in dual streams. Then, with the guidance of variable vocabularies, QGFF integrates specialist and generalist features, enabling them to complement each other. Finally, RIPD is proposed to aggregate multi-source features for more accurate mask predictions. Experiments show that our method outperforms other methods by a large margin, and our proposed LandDiscover50K improves the performance of OVRSISS methods. The proposed dataset and method will be made publicly available at https://github.com/yecy749/GSNet.