# Cross-Modal Bidirectional Interaction Model for Referring Remote Sensing Image Segmentation

arXiv: https://arxiv.org/abs/2410.08613 · HTML: https://mlsanity.com/paper/2410.08613/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 68 references that are in the map.
- First posted on arXiv 11 Oct 2024.

## Builds on (most cited first, 30 of 35)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [MAttNet: Modular Attention Network for Referring Expression Comprehension](https://mlsanity.com/paper/1801.08186/) (2018)
- [CRIS: CLIP-Driven Referring Image Segmentation](https://mlsanity.com/paper/2111.15174/) (2021)
- [LAVT: Language-Aware Vision Transformer for Referring Image Segmentation](https://mlsanity.com/paper/2112.02244/) (2021)
- [Object Detection in Optical Remote Sensing Images: A Survey and A New Benchmark](https://mlsanity.com/paper/1909.00133/) (2019)
- [Remote Sensing Image Change Detection with Transformers](https://mlsanity.com/paper/2103.00208/) (2021)
- [Cross-Modal Self-Attention Network for Referring Image Segmentation](https://mlsanity.com/paper/1904.04745/) (2019)
- [Segmentation from Natural Language Expressions](https://mlsanity.com/paper/1603.06180/) (2016)
- [Object Detection in Aerial Images: A Large-Scale Benchmark and Challenges](https://mlsanity.com/paper/2102.12219/) (2021)
- [Dice Loss for Data-imbalanced NLP Tasks](https://mlsanity.com/paper/1911.02855/) (2019)
- [Referring Image Segmentation via Cross-Modal Progressive Comprehension](https://mlsanity.com/paper/2010.00514/) (2020)
- [RSVG: Exploring Data and Models for Visual Grounding on Remote Sensing Data](https://mlsanity.com/paper/2210.12634/) (2022)
- [VRSBench: A Versatile Vision-Language Benchmark Dataset for Remote Sensing Image Understanding](https://mlsanity.com/paper/2406.12384/) (2024)
- [PhraseCut: Language-based Image Segmentation in the Wild](https://mlsanity.com/paper/2008.01187/) (2020)
- [Linguistic Structure Guided Context Modeling for Referring Image Segmentation](https://mlsanity.com/paper/2010.00515/) (2020)
- [Cross-Modal Progressive Comprehension for Referring Segmentation](https://mlsanity.com/paper/2105.07175/) (2021)
- [Rotated Multi-Scale Interaction Network for Referring Remote Sensing Image Segmentation](https://mlsanity.com/paper/2312.12470/) (2023)
- [Beyond One-to-One: Rethinking the Referring Image Segmentation](https://mlsanity.com/paper/2308.13853/) (2023)
- [Bridging Vision and Language Encoders: Parameter-Efficient Tuning for Referring Image Segmentation](https://mlsanity.com/paper/2307.11545/) (2023)
- [RRSIS: Referring Remote Sensing Image Segmentation](https://mlsanity.com/paper/2306.08625/) (2023)
- [Poly Kernel Inception Network for Remote Sensing Detection](https://mlsanity.com/paper/2403.06258/) (2024)
- [Towards Robust Referring Image Segmentation](https://mlsanity.com/paper/2209.09554/) (2022)
- [Remote Sensing Cross-Modal Text-Image Retrieval Based on Global and Local Information](https://mlsanity.com/paper/2204.09860/) (2022)
- [From Easy to Hard: Learning Language-guided Curriculum for Visual Question Answering on Remote Sensing Data](https://mlsanity.com/paper/2205.03147/) (2022)
- [Txt2Img-MHN: Remote Sensing Image Generation from Text Using Modern Hopfield Networks](https://mlsanity.com/paper/2208.04441/) (2022)

## Built on by (most cited first, 26 of 26)

- [SegEarth-R1: Geospatial Pixel Reasoning via Large Language Model](https://mlsanity.com/paper/2504.09644/) (2025)
- [RemoteSAM: Towards Segment Anything for Earth Observation](https://mlsanity.com/paper/2505.18022/) (2025)
- [RSRefSeg: Referring Remote Sensing Image Segmentation with Foundation Models](https://mlsanity.com/paper/2501.06809/) (2025)
- [EarthMind: Leveraging Cross-Sensor Data for Advanced Earth Observation Interpretation with a Unified Multimodal LLM](https://mlsanity.com/paper/2506.01667/) (2025)
- [RSRefSeg 2: Decoupling Referring Remote Sensing Image Segmentation with Foundation Models](https://mlsanity.com/paper/2507.06231/) (2025)
- [UniGeoSeg: Towards Unified Open-World Segmentation for Geospatial Scenes](https://mlsanity.com/paper/2511.23332/) (2025)
- [SegEarth-R2: Towards Comprehensive Language-guided Segmentation for Remote Sensing Images](https://mlsanity.com/paper/2512.20013/) (2025)
- [RemoteAgent: Bridging Vague Human Intents and Earth Observation with RL-based Agentic MLLMs](https://mlsanity.com/paper/2604.07765/) (2026)
- [DiffRIS: Enhancing Referring Remote Sensing Image Segmentation with Pre-trained Text-to-Image Diffusion Models](https://mlsanity.com/paper/2506.18946/) (2025)
- [Bridging Semantics and Geometry: A Decoupled LVLM-SAM Framework for Reasoning Segmentation in Optical Remote Sensing](https://mlsanity.com/paper/2512.19302/) (2025)
- [AeroReformer: Aerial Referring Transformer for UAV-based Referring Image Segmentation](https://mlsanity.com/paper/2502.16680/) (2025)
- [Earth-OneVision: Extending Remote Sensing Multimodal Large Language Models to More Sensor Modalities and Tasks](https://mlsanity.com/paper/2606.10819/) (2026)
- [Referring Remote Sensing Image Segmentation with Cross-view Semantics Interaction Network](https://mlsanity.com/paper/2508.01331/) (2025)
- [RIS-LAD: A Benchmark and Model for Referring Low-Altitude Drone Image Segmentation](https://mlsanity.com/paper/2507.20920/) (2025)
- [Breaking the Box: Enhancing Remote Sensing Image Segmentation with Freehand Sketches](https://mlsanity.com/paper/2503.12191/) (2025)
- [From Pixels to Images: A Structural Survey of Deep Learning Paradigms in Remote Sensing Image Semantic Segmentation](https://mlsanity.com/paper/2505.15147/) (2025)
- [Understanding What Is Not Said:Referring Remote Sensing Image Segmentation with Scarce Expressions](https://mlsanity.com/paper/2510.22760/) (2025)
- [CroBIM-V: Memory-Quality Controlled Remote Sensing Referring Video Object Segmentation](https://mlsanity.com/paper/2601.12076/) (2026)
- [DARC-CLIP: Dynamic Adaptive Refinement with Cross-Attention for Meme Understanding](https://mlsanity.com/paper/2604.23214/) (2026)
- [Training-Free Open-Vocabulary Visual Grounding for Remote Sensing Images and Videos](https://mlsanity.com/paper/2606.16124/) (2026)
- [GeoSelect: Spatial-Program Execution for Training-Free Referring Remote Sensing Image Segmentation](https://mlsanity.com/paper/2607.03869/) (2026)
- [AeroReformer2: Spoken-Query Referring Segmentation for Aerial Images](https://mlsanity.com/paper/2608.08874/) (2026)
- [DiCoR: Decoupled Referent Disambiguation and Contour Recalibration for Efficient Referring Remote Sensing Image Segmentation](https://mlsanity.com/paper/2608.12980/) (2026)
- [FIRM: Fine-Grained Intra-Token Representation of Masks for Remote Sensing Reasoning Segmentation](https://mlsanity.com/paper/2608.13980/) (2026)
- [VPRef: A Cross-Domain Benchmark for Referring Remote Sensing Image Segmentation](https://mlsanity.com/paper/2609.16486/) (2026)
- [GeoRefer-Bench: A Benchmark from Referring Pixels to Verifiable Geospatial Reasoning](https://mlsanity.com/paper/2609.29541/) (2026)

## Abstract (arXiv)

Given a natural language expression and a remote sensing image, the goal of referring remote sensing image segmentation (RRSIS) is to generate a pixel-level mask of the target object identified by the referring expression. In contrast to natural scenarios, expressions in RRSIS often involve complex geospatial relationships, with target objects of interest that vary significantly in scale and lack visual saliency, thereby increasing the difficulty of achieving precise segmentation. To address the aforementioned challenges, a novel RRSIS framework is proposed, termed the cross-modal bidirectional interaction model (CroBIM). Specifically, a context-aware prompt modulation (CAPM) module is designed to integrate spatial positional relationships and task-specific knowledge into the linguistic features, thereby enhancing the ability to capture the target object. Additionally, a language-guided feature aggregation (LGFA) module is introduced to integrate linguistic information into multi-scale visual features, incorporating an attention deficit compensation mechanism to enhance feature aggregation. Finally, a mutual-interaction decoder (MID) is designed to enhance cross-modal feature alignment through cascaded bidirectional cross-attention, thereby enabling precise segmentation mask prediction. To further forster the research of RRSIS, we also construct RISBench, a new large-scale benchmark dataset comprising 52,472 image-language-label triplets. Extensive benchmarking on RISBench and two other prevalent datasets demonstrates the superior performance of the proposed CroBIM over existing state-of-the-art (SOTA) methods. The source code for CroBIM and the RISBench dataset will be publicly available at https://github.com/HIT-SIRS/CroBIM