mlsanityData to 7 Oct 2026

Rotated Multi-Scale Interaction Network for Referring Remote Sensing Image Segmentation

arXiv 2312.12470 ↗First posted 19 Dec 2023cs.CVcited by 76

Not on the map. Why?

Cited by 21 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Referring Remote Sensing Image Segmentation (RRSIS) is a new challenge that combines computer vision and natural language processing, delineating specific regions in aerial images as described by textual queries. Traditional Referring Image Segmentation (RIS) approaches have been impeded by the complex spatial scales and orientations found in aerial imagery, leading to suboptimal segmentation results. To address these challenges, we introduce the Rotated Multi-Scale Interaction Network (RMSIN), an innovative approach designed for the unique demands of RRSIS. RMSIN incorporates an Intra-scale Interaction Module (IIM) to effectively address the fine-grained detail required at multiple scales and a Cross-scale Interaction Module (CIM) for integrating these details coherently across the network. Furthermore, RMSIN employs an Adaptive Rotated Convolution (ARC) to account for the diverse orientations of objects, a novel contribution that significantly enhances segmentation accuracy. To assess the efficacy of RMSIN, we have curated an expansive dataset comprising 17,402 image-caption-mask triplets, which is unparalleled in terms of scale and variety. This dataset not only presents the model with a wide range of spatial and rotational scenarios but also establishes a stringent benchmark for the RRSIS task, ensuring a rigorous evaluation of performance. Our experimental evaluations demonstrate the exceptional performance of RMSIN, surpassing existing state-of-the-art models by a significant margin. All datasets and code are made available at https://github.com/Lsan2401/RMSIN.

Key facts. Cited by 76 papers in the map (data through 7 Oct 2026). Builds on 51 of its 66 references that are in the map. First posted on arXiv 19 Dec 2023.

Builds on

51 of 66 references.

References · 51

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  6. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  7. 2016Modeling Context in Referring Expressionscited by 1,054
  8. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 897
  9. 2021MDETR -- Modulated Detection for End-to-End Multi-Modal Understandingcited by 811
  10. 2021CRIS: CLIP-Driven Referring Image Segmentationcited by 353
  11. 2021LAVT: Language-Aware Vision Transformer for Referring Image Segmentationcited by 347
  12. 2022X-CLIP: End-to-End Multi-grained Contrastive Learning for Video-Text Retrievalcited by 279
38 more
  1. 2019Tackling Climate Change with Machine Learningcited by 264
  2. 2019Cross-Modal Self-Attention Network for Referring Image Segmentationcited by 263
  3. 2016Segmentation from Natural Language Expressionscited by 257
  4. 2016Modeling Context Between Objects for Referring Expression Understandingcited by 250
  5. 2021Vision-Language Transformer and Query Generation for Referring Segmentationcited by 239
  6. 2020Multi-task Collaborative Network for Joint Referring Expression Comprehension and Segmentationcited by 204
  7. 2019A Fast and Accurate One-Stage Approach to Visual Groundingcited by 194
  8. 2021Referring Transformer: A One-step Approach to Multi-task Visual Groundingcited by 143
  9. 2022SeqTR: A Simple yet Universal Network for Visual Groundingcited by 138
  10. 2019Gliding vertex on the horizontal bounding box for multi-oriented object detectioncited by 134
  11. 2020Referring Image Segmentation via Cross-Modal Progressive Comprehensioncited by 131
  12. 2021Learning High-Precision Bounding Box for Rotated Object Detection via Kullback-Leibler Divergencecited by 130
  13. 2017Recurrent Multimodal Interaction for Referring Image Segmentationcited by 125
  14. 2022RSVG: Exploring Data and Models for Visual Grounding on Remote Sensing Datacited by 121
  15. 2020PhraseCut: Language-based Image Segmentation in the Wildcited by 116
  16. 2022ReSTR: Convolution-free Referring Image Segmentation Using Transformerscited by 112
  17. 2019A Real-Time Cross-modality Correlation Filtering Method for Referring Expression Comprehensioncited by 111
  18. 2016Modeling Relationships in Referential Expressions with Compositional Modular Networkscited by 106
  19. 2020Linguistic Structure Guided Context Modeling for Referring Image Segmentationcited by 102
  20. 2021Cross-Modal Progressive Comprehension for Referring Segmentationcited by 102
  21. 2019Learning to Compose and Reason with Language Tree Structures for Visual Groundingcited by 98
  22. 2021Locate then Segment: A Strong Pipeline for Referring Image Segmentationcited by 98
  23. 2021Multi-Scale High-Resolution Vision Transformer for Semantic Segmentationcited by 92
  24. 2018Dynamic Multimodal Instance Segmentation guided by natural language queriescited by 83
  25. 2017Grounding Referring Expressions in Images by Variational Contextcited by 82
  26. 2022The KFIoU Loss for Rotated Object Detectioncited by 73
  27. 2021Dual-Level Collaborative Transformer for Image Captioningcited by 71
  28. 2017Parallel Attention: A Unified Framework for Visual Object Discovery through Dialogs and Queriescited by 56
  29. 2023Adaptive Rotated Convolution for Rotated Object Detectioncited by 51
  30. 2022Towards Robust Referring Image Segmentationcited by 49
  31. 2019Referring Expression Object Segmentation with Caption-Aware Consistencycited by 46
  32. 2022X-DETR: A Versatile Architecture for Instance-wise Vision-Language Taskscited by 41
  33. 2023Multi-Modal Mutual Attention and Iterative Interaction for Referring Image Segmentationcited by 36
  34. 20233D-STMN: Dependency-Driven Superpoint-Text Matching Network for End-to-End 3D Referring Expression Segmentationcited by 36
  35. 2023Towards Local Visual Modeling for Image Captioningcited by 26
  36. 2023Shatter and Gather: Learning Referring Image Segmentation with Text Supervisioncited by 25
  37. 2022Towards Lightweight Transformer via Group-wise Transformation for Vision-and-Language Taskscited by 21
  38. 2021Towards Language-guided Visual Recognition via Dynamic Convolutionscited by 17

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 14; 2025: 36; 2026: 26.

76 papers.

Sort:

By area:

  1. 2024VRSBench: A Versatile Vision-Language Benchmark Dataset for Remote Sensing Image Understandingcited by 116
  2. 2024Toward Visual Grounding: A Surveycited by 52
  3. 2024GEOBench-VLM: Benchmarking Vision-Language Models for Geospatial Taskscited by 50
  4. 2025XLRS-Bench: Could Your Multimodal LLMs Understand Extremely Large Ultra-High-Resolution Remote Sensing Imagery?cited by 50
  5. 2025RemoteSAM: Towards Segment Anything for Earth Observationcited by 38
  6. 2024Towards Vision-Language Geo-Foundation Model: A Surveycited by 36
  7. 2025When Large Vision-Language Model Meets Large Remote Sensing Imagery: Coarse-to-Fine Text-Guided Token Pruningcited by 35
  8. 2025DisasterM3: A Remote Sensing Vision-Language Dataset for Disaster Damage Assessment and Responsecited by 35
  9. 2024Cross-Modal Bidirectional Interaction Model for Referring Remote Sensing Image Segmentationcited by 26
  10. 2024Exploring Fine-Grained Image-Text Alignment for Referring Remote Sensing Image Segmentationcited by 24
  11. 2025RSRefSeg: Referring Remote Sensing Image Segmentation with Foundation Modelscited by 22
  12. 2025Multimodal Referring Segmentation: A Surveycited by 22
38 more
  1. 2025Geo-R1: Improving Few-Shot Geospatial Referring Expression Understanding with Reinforcement Fine-Tuningcited by 19
  2. 2025EarthMind: Leveraging Cross-Sensor Data for Advanced Earth Observation Interpretation with a Unified Multimodal LLMcited by 18
  3. 2025RSRefSeg 2: Decoupling Referring Remote Sensing Image Segmentation with Foundation Modelscited by 17
  4. 2024Show Me What and Where has Changed? Question Answering and Grounding for Remote Sensing Change Detectioncited by 15
  5. 2025Hybrid Global-Local Representation with Augmented Spatial Guidance for Zero-Shot Referring Image Segmentationcited by 14
  6. 2024Pseudo-RIS: Distinctive Pseudo-supervision Generation for Referring Image Segmentationcited by 13
  7. 2025A Large-Scale Referring Remote Sensing Image Segmentation Dataset and Benchmarkcited by 12
  8. 2025GeoPix: Multi-Modal Large Language Model for Pixel-level Image Understanding in Remote Sensingcited by 10
  9. 2025UniGeoSeg: Towards Unified Open-World Segmentation for Geospatial Scenescited by 10
  10. 2025CADFormer: Fine-Grained Cross-modal Alignment and Decoding Transformer for Referring Remote Sensing Image Segmentationcited by 9
  11. 2025InstructSAM: A Training-Free Framework for Instruction-Oriented Remote Sensing Object Recognitioncited by 8
  12. 2025Text4Seg++: Advancing Image Segmentation via Generative Language Modelingcited by 8
  13. 2025Annotation-Free Open-Vocabulary Segmentation for Remote-Sensing Imagescited by 7
  14. 2026RemoteAgent: Bridging Vague Human Intents and Earth Observation with RL-based Agentic MLLMscited by 7
  15. 2024ImageRAG: Enhancing Ultra High Resolution Remote Sensing Imagery Analysis with ImageRAGcited by 6
  16. 2025GeoRSMLLM: A Multimodal Large Language Model for Vision-Language Tasks in Geoscience and Remote Sensingcited by 6
  17. 2025DiffRIS: Enhancing Referring Remote Sensing Image Segmentation with Pre-trained Text-to-Image Diffusion Modelscited by 6
  18. 2025AeroReformer: Aerial Referring Transformer for UAV-based Referring Image Segmentationcited by 5
  19. 2025Multimodal-Aware Fusion Network for Referring Remote Sensing Image Segmentationcited by 5
  20. 2026CroBIM-U: Uncertainty-Driven Referring Remote Sensing Image Segmentationcited by 5
  21. 2024UEMM-Air: Make Unmanned Aerial Vehicles Perform More Multi-modal Taskscited by 4
  22. 2025RS2-SAM2: Customized SAM2 for Referring Remote Sensing Image Segmentationcited by 4
  23. 2025EarthScape: A Multimodal Dataset for Surficial Geologic Mapping and Earth Surface Analysiscited by 4
  24. 2026DVGBench: Implicit-to-Explicit Visual Grounding Benchmark in UAV Imagery with Large Vision-Language Modelscited by 4
  25. 2024Image Captioning via Dynamic Path Customizationcited by 3
  26. 2025Mamba-MOC: A Multicategory Remote Object Counting via State Space Modelcited by 3
  27. 2026Earth-OneVision: Extending Remote Sensing Multimodal Large Language Models to More Sensor Modalities and Taskscited by 3
  28. 2024Embedding Generalized Semantic Knowledge into Few-Shot Remote Sensing Segmentationcited by 2
  29. 2025Referring Remote Sensing Image Segmentation with Cross-view Semantics Interaction Networkcited by 2
  30. 2025Generalized Referring Expression Segmentation on Aerial Photoscited by 2
  31. 2026Enabling Training-Free Text-Based Remote Sensing Segmentationcited by 2
  32. 2026GeoAlignCLIP: Enhancing Fine-Grained Vision-Language Alignment in Remote Sensing via Multi-Granular Consistency Learningcited by 2
  33. 2024Harnessing Vision-Language Pretrained Models with Temporal-Aware Adaptation for Referring Video Object Segmentationcited by 1
  34. 2024Advancements in Visual Language Models for Remote Sensing: Datasets, Capabilities, and Enhancement Techniquescited by 1
  35. 2025RIS-LAD: A Benchmark and Model for Referring Low-Altitude Drone Image Segmentationcited by 1
  36. 2025Small Lesions-aware Bidirectional Multimodal Multiscale Fusion Network for Lung Disease Classificationcited by 1
  37. 2025Remote Sensing Image Intelligent Interpretation with the Language-Centered Perspective: Principles, Methods and Challengescited by 1
  38. 2025Label-Efficient Hyperspectral Image Classification via Spectral FiLM Modulation of Low-Level Pretrained Diffusion Featurescited by 1

50 most cited of 76 All 76 (JSON)