mlsanityData to 7 Oct 2026

Cross-Modal Self-Attention Network for Referring Image Segmentation

arXiv 1904.04745 ↗First posted 9 Apr 2019cs.CV cs.CLcited by 263

Not on the map. Why?

Abstract (arXiv)

We consider the problem of referring image segmentation. Given an input image and a natural language expression, the goal is to segment the object referred by the language expression in the image. Existing works in this area treat the language expression and the input image separately in their representations. They do not sufficiently capture long-range correlations between these two modalities. In this paper, we propose a cross-modal self-attention (CMSA) module that effectively captures the long-range dependencies between linguistic and visual features. Our model can adaptively focus on informative words in the referring expression and important regions in the input image. In addition, we propose a gated multi-level fusion module to selectively integrate self-attentive cross-modal features corresponding to different levels in the image. This module controls the information flow of features at different levels. We validate the proposed approach on four evaluation datasets. Our proposed approach consistently outperforms existing state-of-the-art methods.

Key facts. Cited by 263 papers in the map (data through 7 Oct 2026). First posted on arXiv 9 Apr 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 50; 2022: 45; 2023: 71; 2024: 46; 2025: 34; 2026: 17.

263 papers.

Sort:

By area:

  1. 2021Transformers in Vision: A Surveycited by 1,009
  2. 2023Segment Everything Everywhere All at Oncecited by 546
  3. 2021Image Segmentation Using Text and Image Promptscited by 509
  4. 2021Scaling Open-Vocabulary Image Segmentation with Image-Level Labelscited by 471
  5. 2021Attention Bottlenecks for Multimodal Fusioncited by 361
  6. 2021CRIS: CLIP-Driven Referring Image Segmentationcited by 353
  7. 2021LAVT: Language-Aware Vision Transformer for Referring Image Segmentationcited by 347
  8. 2022Generalized Decoding for Pixel, Image, and Languagecited by 292
  9. 20213D Human Pose Estimation with Spatial and Temporal Transformerscited by 271
  10. 2021Generalizing Face Forgery Detection with High-frequency Featurescited by 261
  11. 2023GRES: Generalized Referring Expression Segmentationcited by 252
  12. 2021Vision-Language Transformer and Query Generation for Referring Segmentationcited by 239
38 more
  1. 2023MeViS: A Large-scale Benchmark for Video Segmentation with Motion Expressionscited by 225
  2. 2021ATISS: Autoregressive Transformers for Indoor Scene Synthesiscited by 195
  3. 2023Universal Instance Perception as Object Discovery and Retrievalcited by 190
  4. 2022Are Multimodal Transformers Robust to Missing Modality?cited by 183
  5. 2022Language as Queries for Referring Video Object Segmentationcited by 180
  6. 2023GSVA: Generalized Segmentation via Multimodal Large Language Modelscited by 153
  7. 2023PolyFormer: Referring Image Segmentation as Sequential Polygon Generationcited by 147
  8. 2021Referring Transformer: A One-step Approach to Multi-task Visual Groundingcited by 143
  9. 2022SeqTR: A Simple yet Universal Network for Visual Groundingcited by 138
  10. 2024Mobile Edge Intelligence for Large Language Models: A Contemporary Surveycited by 125
  11. 2022ReSTR: Convolution-free Referring Image Segmentation Using Transformerscited by 112
  12. 2021Encoder Fusion Network with Co-Attention Embedding for Referring Image Segmentationcited by 111
  13. 2022VLT: Vision-Language Transformer and Query Generation for Referring Segmentationcited by 110
  14. 2023Language Prompt for Autonomous Drivingcited by 110
  15. 2023Foundational Models Defining a New Era in Vision: A Survey and Outlookcited by 109
  16. 2021Cross-Modal Progressive Comprehension for Referring Segmentationcited by 102
  17. 2024One Token to Seg Them All: Language Instructed Reasoning Segmentation in Videoscited by 99
  18. 2021Locate then Segment: A Strong Pipeline for Referring Image Segmentationcited by 98
  19. 2024Neural Plasticity-Inspired Multimodal Foundation Model for Earth Observationcited by 97
  20. 2022Cross-modal Map Learning for Vision and Language Navigationcited by 91
  21. 2023Referring Multi-Object Trackingcited by 91
  22. 2021LanguageRefer: Spatial-Language Model for 3D Visual Groundingcited by 88
  23. 2021Cross-Modality Fusion Transformer for Multispectral Object Detectioncited by 86
  24. 2021U-Net Transformer: Self and Cross Attention for Medical Image Segmentationcited by 81
  25. 2021Free-form Description Guided 3D Visual Graph Network for Object Grounding in Point Cloudcited by 81
  26. 2023Rotated Multi-Scale Interaction Network for Referring Remote Sensing Image Segmentationcited by 76
  27. 2023General Object Foundation Model for Images and Videos at Scalecited by 71
  28. 2024EVF-SAM: Early Vision-Language Fusion for Text-Prompted Segment Anything Modelcited by 63
  29. 2023OnlineRefer: A Simple Online Baseline for Referring Video Object Segmentationcited by 62
  30. 2023Zero-shot Referring Image Segmentation with Global-Local Context Featurescited by 61
  31. 2023Bridging Vision and Language Encoders: Parameter-Efficient Tuning for Referring Image Segmentationcited by 61
  32. 2023RRSIS: Referring Remote Sensing Image Segmentationcited by 59
  33. 2023SOC: Semantic-Assisted Object Cluster for Referring Video Object Segmentationcited by 57
  34. 2023C2Former: Calibrated and Complementary Transformer for RGB-Infrared Object Detectioncited by 57
  35. 2024DoraemonGPT: Toward Understanding Dynamic Scenes with Large Language Models (Exemplified as A Video Agent)cited by 56
  36. 2022HyperTransformer: Model Generation for Supervised and Semi-Supervised Few-Shot Learningcited by 55
  37. 2024Decoupling Static and Hierarchical Motion Perception for Referring Video Segmentationcited by 55
  38. 2022Unsupervised Semantic Segmentation with Self-supervised Object-centric Representationscited by 54

50 most cited of 263 All 263 (JSON)