# RSRefSeg: Referring Remote Sensing Image Segmentation with Foundation Models

arXiv: https://arxiv.org/abs/2501.06809 · HTML: https://mlsanity.com/paper/2501.06809/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 13 references that are in the map.
- First posted on arXiv 12 Jan 2025.

## Builds on (most cited first, 11 of 11)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [LAVT: Language-Aware Vision Transformer for Referring Image Segmentation](https://mlsanity.com/paper/2112.02244/) (2021)
- [Cross-Modal Self-Attention Network for Referring Image Segmentation](https://mlsanity.com/paper/1904.04745/) (2019)
- [Referring Image Segmentation via Cross-Modal Progressive Comprehension](https://mlsanity.com/paper/2010.00514/) (2020)
- [Linguistic Structure Guided Context Modeling for Referring Image Segmentation](https://mlsanity.com/paper/2010.00515/) (2020)
- [Cross-Modal Progressive Comprehension for Referring Segmentation](https://mlsanity.com/paper/2105.07175/) (2021)
- [Rotated Multi-Scale Interaction Network for Referring Remote Sensing Image Segmentation](https://mlsanity.com/paper/2312.12470/) (2023)
- [RRSIS: Referring Remote Sensing Image Segmentation](https://mlsanity.com/paper/2306.08625/) (2023)
- [Cross-Modal Bidirectional Interaction Model for Referring Remote Sensing Image Segmentation](https://mlsanity.com/paper/2410.08613/) (2024)
- [Cross-aware Early Fusion with Stage-divided Vision and Language Transformer Encoders for Referring Image Segmentation](https://mlsanity.com/paper/2408.07539/) (2024)

## Built on by (most cited first, 22 of 22)

- [RSRefSeg 2: Decoupling Referring Remote Sensing Image Segmentation with Foundation Models](https://mlsanity.com/paper/2507.06231/) (2025)
- [UniGeoSeg: Towards Unified Open-World Segmentation for Geospatial Scenes](https://mlsanity.com/paper/2511.23332/) (2025)
- [DiffRIS: Enhancing Referring Remote Sensing Image Segmentation with Pre-trained Text-to-Image Diffusion Models](https://mlsanity.com/paper/2506.18946/) (2025)
- [Agentic AI in Remote Sensing: Foundations, Taxonomy, and Emerging Systems](https://mlsanity.com/paper/2601.01891/) (2026)
- [AeroReformer: Aerial Referring Transformer for UAV-based Referring Image Segmentation](https://mlsanity.com/paper/2502.16680/) (2025)
- [CroBIM-U: Uncertainty-Driven Referring Remote Sensing Image Segmentation](https://mlsanity.com/paper/2601.03490/) (2026)
- [GRASP: Geospatial pixel Reasoning viA Structured Policy learning](https://mlsanity.com/paper/2508.17102/) (2025)
- [SeG-SR: Integrating Semantic Knowledge into Remote Sensing Image Super-Resolution via Vision-Language Model](https://mlsanity.com/paper/2505.23010/) (2025)
- [Urban Socio-Semantic Segmentation with Vision-Language Reasoning](https://mlsanity.com/paper/2601.10477/) (2026)
- [Generalized Referring Expression Segmentation on Aerial Photos](https://mlsanity.com/paper/2512.07338/) (2025)
- [Enabling Training-Free Text-Based Remote Sensing Segmentation](https://mlsanity.com/paper/2602.17799/) (2026)
- [RIS-LAD: A Benchmark and Model for Referring Low-Altitude Drone Image Segmentation](https://mlsanity.com/paper/2507.20920/) (2025)
- [Remote Sensing Image Intelligent Interpretation with the Language-Centered Perspective: Principles, Methods and Challenges](https://mlsanity.com/paper/2508.06832/) (2025)
- [Vision-Language Model Purified Semi-Supervised Semantic Segmentation for Remote Sensing Images](https://mlsanity.com/paper/2602.00202/) (2026)
- [An Open-Source Benchmark and Baseline for Multi-temporal Referring Segmentation](https://mlsanity.com/paper/2606.00987/) (2026)
- [CROSS: Cascaded Distillation and Dual-Constraint Grounding for Remote Sensing Referring Segmentation](https://mlsanity.com/paper/2608.03147/) (2026)
- [Breaking the Box: Enhancing Remote Sensing Image Segmentation with Freehand Sketches](https://mlsanity.com/paper/2503.12191/) (2025)
- [DGL-RSIS: Decoupling Global Spatial Context and Local Class Semantics for Training-Free Remote Sensing Image Segmentation](https://mlsanity.com/paper/2509.00598/) (2025)
- [Understanding What Is Not Said:Referring Remote Sensing Image Segmentation with Scarce Expressions](https://mlsanity.com/paper/2510.22760/) (2025)
- [An Efficient and Effective Encoder Model for Vision and Language Tasks in the Remote Sensing Domain](https://mlsanity.com/paper/2512.15531/) (2025)
- [Semantic-Driven Scale and Spatial Selection for Efficient Cross-Modal Alignment in Referring Remote Sensing Image Segmentation](https://mlsanity.com/paper/2606.30244/) (2026)
- [DiCoR: Decoupled Referent Disambiguation and Contour Recalibration for Efficient Referring Remote Sensing Image Segmentation](https://mlsanity.com/paper/2608.12980/) (2026)

## Abstract (arXiv)

Referring remote sensing image segmentation is crucial for achieving fine-grained visual understanding through free-format textual input, enabling enhanced scene and object extraction in remote sensing applications. Current research primarily utilizes pre-trained language models to encode textual descriptions and align them with visual modalities, thereby facilitating the expression of relevant visual features. However, these approaches often struggle to establish robust alignments between fine-grained semantic concepts, leading to inconsistent representations across textual and visual information. To address these limitations, we introduce a referring remote sensing image segmentation foundational model, RSRefSeg. RSRefSeg leverages CLIP for visual and textual encoding, employing both global and local textual semantics as filters to generate referring-related visual activation features in the latent space. These activated features then serve as input prompts for SAM, which refines the segmentation masks through its robust visual generalization capabilities. Experimental results on the RRSIS-D dataset demonstrate that RSRefSeg outperforms existing methods, underscoring the effectiveness of foundational models in enhancing multimodal task comprehension. The code is available at urlhttps://github.com/KyanChen/RSRefSeg.