# Exploring Fine-Grained Image-Text Alignment for Referring Remote Sensing Image Segmentation

arXiv: https://arxiv.org/abs/2409.13637 · HTML: https://mlsanity.com/paper/2409.13637/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 44 references that are in the map.
- First posted on arXiv 20 Sep 2024.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [LAVT: Language-Aware Vision Transformer for Referring Image Segmentation](https://mlsanity.com/paper/2112.02244/) (2021)
- [GeoChat: Grounded Large Vision-Language Model for Remote Sensing](https://mlsanity.com/paper/2311.15826/) (2023)
- [Cross-Modal Self-Attention Network for Referring Image Segmentation](https://mlsanity.com/paper/1904.04745/) (2019)
- [Segmentation from Natural Language Expressions](https://mlsanity.com/paper/1603.06180/) (2016)
- [Referring Image Segmentation via Cross-Modal Progressive Comprehension](https://mlsanity.com/paper/2010.00514/) (2020)
- [Recurrent Multimodal Interaction for Referring Image Segmentation](https://mlsanity.com/paper/1703.07939/) (2017)
- [RSVG: Exploring Data and Models for Visual Grounding on Remote Sensing Data](https://mlsanity.com/paper/2210.12634/) (2022)
- [ReSTR: Convolution-free Referring Image Segmentation Using Transformers](https://mlsanity.com/paper/2203.16768/) (2022)
- [Linguistic Structure Guided Context Modeling for Referring Image Segmentation](https://mlsanity.com/paper/2010.00515/) (2020)
- [Cross-Modal Progressive Comprehension for Referring Segmentation](https://mlsanity.com/paper/2105.07175/) (2021)
- [Locate then Segment: A Strong Pipeline for Referring Image Segmentation](https://mlsanity.com/paper/2103.16284/) (2021)
- [Dynamic Multimodal Instance Segmentation guided by natural language queries](https://mlsanity.com/paper/1807.02257/) (2018)
- [Rotated Multi-Scale Interaction Network for Referring Remote Sensing Image Segmentation](https://mlsanity.com/paper/2312.12470/) (2023)
- [RRSIS: Referring Remote Sensing Image Segmentation](https://mlsanity.com/paper/2306.08625/) (2023)
- [Multi-Modal Mutual Attention and Iterative Interaction for Referring Image Segmentation](https://mlsanity.com/paper/2305.15302/) (2023)
- [Cross-aware Early Fusion with Stage-divided Vision and Language Transformer Encoders for Referring Image Segmentation](https://mlsanity.com/paper/2408.07539/) (2024)

## Built on by (most cited first, 24 of 24)

- [RemoteSAM: Towards Segment Anything for Earth Observation](https://mlsanity.com/paper/2505.18022/) (2025)
- [RSRefSeg 2: Decoupling Referring Remote Sensing Image Segmentation with Foundation Models](https://mlsanity.com/paper/2507.06231/) (2025)
- [Intra- and Cross-Scale Feature Interaction Network for Referring Remote Sensing Image Segmentation](https://mlsanity.com/paper/2506.03583/) (2025)
- [SegEarth-R2: Towards Comprehensive Language-guided Segmentation for Remote Sensing Images](https://mlsanity.com/paper/2512.20013/) (2025)
- [CADFormer: Fine-Grained Cross-modal Alignment and Decoding Transformer for Referring Remote Sensing Image Segmentation](https://mlsanity.com/paper/2503.23456/) (2025)
- [AeroReformer: Aerial Referring Transformer for UAV-based Referring Image Segmentation](https://mlsanity.com/paper/2502.16680/) (2025)
- [RS2-SAM2: Customized SAM2 for Referring Remote Sensing Image Segmentation](https://mlsanity.com/paper/2503.07266/) (2025)
- [SAM-Based Building Change Detection with Distribution-Aware Fourier Adaptation and Edge-Constrained Warping](https://mlsanity.com/paper/2504.12619/) (2025)
- [Referring Remote Sensing Image Segmentation with Cross-view Semantics Interaction Network](https://mlsanity.com/paper/2508.01331/) (2025)
- [Generalized Referring Expression Segmentation on Aerial Photos](https://mlsanity.com/paper/2512.07338/) (2025)
- [Enabling Training-Free Text-Based Remote Sensing Segmentation](https://mlsanity.com/paper/2602.17799/) (2026)
- [RIS-LAD: A Benchmark and Model for Referring Low-Altitude Drone Image Segmentation](https://mlsanity.com/paper/2507.20920/) (2025)
- [An Open-Source Benchmark and Baseline for Multi-temporal Referring Segmentation](https://mlsanity.com/paper/2606.00987/) (2026)
- [CROSS: Cascaded Distillation and Dual-Constraint Grounding for Remote Sensing Referring Segmentation](https://mlsanity.com/paper/2608.03147/) (2026)
- [Constrained Gaussian Process Motion Planning via Stein Variational Newton Inference](https://mlsanity.com/paper/2504.04936/) (2025)
- [Exploring Generalizable Pre-training for Real-world Change Detection via Geometric Estimation](https://mlsanity.com/paper/2504.14306/) (2025)
- [Understanding What Is Not Said:Referring Remote Sensing Image Segmentation with Scarce Expressions](https://mlsanity.com/paper/2510.22760/) (2025)
- [Image-Conditioned Instance Prompt Network for Referring Remote Sensing Image Segmentation](https://mlsanity.com/paper/2605.24532/) (2026)
- [Semantic-Driven Scale and Spatial Selection for Efficient Cross-Modal Alignment in Referring Remote Sensing Image Segmentation](https://mlsanity.com/paper/2606.30244/) (2026)
- [GeoSelect: Spatial-Program Execution for Training-Free Referring Remote Sensing Image Segmentation](https://mlsanity.com/paper/2607.03869/) (2026)
- [AeroReformer2: Spoken-Query Referring Segmentation for Aerial Images](https://mlsanity.com/paper/2608.08874/) (2026)
- [DiCoR: Decoupled Referent Disambiguation and Contour Recalibration for Efficient Referring Remote Sensing Image Segmentation](https://mlsanity.com/paper/2608.12980/) (2026)
- [DeCo: Efficient Decouple-to-Couple Learning for Multi-Task Visual Grounding](https://mlsanity.com/paper/2609.24409/) (2026)
- [GeoRefer-Bench: A Benchmark from Referring Pixels to Verifiable Geospatial Reasoning](https://mlsanity.com/paper/2609.29541/) (2026)

## Abstract (arXiv)

Given a language expression, referring remote sensing image segmentation (RRSIS) aims to identify ground objects and assign pixel-wise labels within the imagery. The one of key challenges for this task is to capture discriminative multi-modal features via text-image alignment. However, the existing RRSIS methods use one vanilla and coarse alignment, where the language expression is directly extracted to be fused with the visual features. In this paper, we argue that a ``fine-grained image-text alignment'' can improve the extraction of multi-modal information. To this point, we propose a new referring remote sensing image segmentation method to fully exploit the visual and linguistic representations. Specifically, the original referring expression is regarded as context text, which is further decoupled into the ground object and spatial position texts. The proposed fine-grained image-text alignment module (FIAM) would simultaneously leverage the features of the input image and the corresponding texts, obtaining better discriminative multi-modal representation. Meanwhile, to handle the various scales of ground objects in remote sensing, we introduce a Text-aware Multi-scale Enhancement Module (TMEM) to adaptively perform cross-scale fusion and intersections. We evaluate the effectiveness of the proposed method on two public referring remote sensing datasets including RefSegRS and RRSIS-D, and our method obtains superior performance over several state-of-the-art methods. The code will be publicly available at https://github.com/Shaosifan/FIANet.