mlsanityData to 7 Oct 2026

Recurrent Multimodal Interaction for Referring Image Segmentation

arXiv 1703.07939 ↗First posted 23 Mar 2017cs.CVcited by 125

Not on the map. Why?

Abstract (arXiv)

In this paper we are interested in the problem of image segmentation given natural language descriptions, i.e. referring expressions. Existing works tackle this problem by first modeling images and sentences independently and then segment images by combining these two types of representations. We argue that learning word-to-image interaction is more native in the sense of jointly modeling two modalities for the image segmentation task, and we propose convolutional multimodal LSTM to encode the sequential interactions between individual words, visual information, and spatial information. We show that our proposed model outperforms the baseline model on benchmark datasets. In addition, we analyze the intermediate output of the proposed multimodal LSTM approach and empirically explain how this approach enforces a more effective word-to-image interaction.

Key facts. Cited by 125 papers in the map (data through 7 Oct 2026). First posted on arXiv 23 Mar 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 22; 2022: 15; 2023: 35; 2024: 23; 2025: 16; 2026: 14.

125 papers.

Sort:

By area:

  1. 2023Segment Everything Everywhere All at Oncecited by 546
  2. 2021Image Segmentation Using Text and Image Promptscited by 509
  3. 2021CRIS: CLIP-Driven Referring Image Segmentationcited by 353
  4. 2021LAVT: Language-Aware Vision Transformer for Referring Image Segmentationcited by 347
  5. 2022Generalized Decoding for Pixel, Image, and Languagecited by 292
  6. 2021Vision-Language Transformer and Query Generation for Referring Segmentationcited by 239
  7. 2023MeViS: A Large-scale Benchmark for Video Segmentation with Motion Expressionscited by 225
  8. 2023GSVA: Generalized Segmentation via Multimodal Large Language Modelscited by 153
  9. 2023PolyFormer: Referring Image Segmentation as Sequential Polygon Generationcited by 147
  10. 2022ReSTR: Convolution-free Referring Image Segmentation Using Transformerscited by 112
  11. 2021Encoder Fusion Network with Co-Attention Embedding for Referring Image Segmentationcited by 111
  12. 2022VLT: Vision-Language Transformer and Query Generation for Referring Segmentationcited by 110
38 more
  1. 2021Cross-Modal Progressive Comprehension for Referring Segmentationcited by 102
  2. 2021Locate then Segment: A Strong Pipeline for Referring Image Segmentationcited by 98
  3. 2025Your Large Vision-Language Model Only Needs A Few Attention Heads For Visual Groundingcited by 78
  4. 2023Rotated Multi-Scale Interaction Network for Referring Remote Sensing Image Segmentationcited by 76
  5. 2023Beyond One-to-One: Rethinking the Referring Image Segmentationcited by 68
  6. 2024EVF-SAM: Early Vision-Language Fusion for Text-Prompted Segment Anything Modelcited by 63
  7. 2023OnlineRefer: A Simple Online Baseline for Referring Video Object Segmentationcited by 62
  8. 2023Zero-shot Referring Image Segmentation with Global-Local Context Featurescited by 61
  9. 2023Bridging Vision and Language Encoders: Parameter-Efficient Tuning for Referring Image Segmentationcited by 61
  10. 2023RRSIS: Referring Remote Sensing Image Segmentationcited by 59
  11. 2024Decoupling Static and Hierarchical Motion Perception for Referring Video Segmentationcited by 55
  12. 2023Contrastive Grouping with Transformer for Referring Image Segmentationcited by 54
  13. 2022Instance-Specific Feature Propagation for Referring Segmentationcited by 45
  14. 2022Local-Global Context Aware Transformer for Language-Guided Video Segmentationcited by 41
  15. 2023Mask Grounding for Referring Image Segmentationcited by 40
  16. 2024ReMamber: Referring Image Segmentation with Mamba Twistercited by 38
  17. 2023Multi-Modal Mutual Attention and Iterative Interaction for Referring Image Segmentationcited by 36
  18. 2023Referring Image Segmentation Using Text Supervisioncited by 35
  19. 2022Peekaboo: Text to Image Diffusion Models are Zero-Shot Segmentorscited by 34
  20. 2025MeViS: A Multi-Modal Dataset for Referring Motion Expression Video Segmentationcited by 34
  21. 2023LD-ZNet: A Latent Diffusion Approach for Text-Based Image Segmentationcited by 29
  22. 2023Universal Segmentation at Arbitrary Granularity with Language Instructioncited by 29
  23. 2021Referring Segmentation in Images and Videos with Cross-Modal Self-Attention Networkcited by 28
  24. 2021Panoptic Narrative Groundingcited by 27
  25. 2024Cross-Modal Conditioned Reconstruction for Language-guided Medical Image Segmentationcited by 27
  26. 2023Semantic Image Segmentation: Two Decades of Researchcited by 26
  27. 2021Segmentation-grounded Scene Graph Generationcited by 25
  28. 2023Shatter and Gather: Learning Referring Image Segmentation with Text Supervisioncited by 25
  29. 2024Exploring Fine-Grained Image-Text Alignment for Referring Remote Sensing Image Segmentationcited by 24
  30. 2024Curriculum Point Prompting for Weakly-Supervised Referring Image Segmentationcited by 23
  31. 2022Referring Image Mattingcited by 22
  32. 2025Multimodal Referring Segmentation: A Surveycited by 22
  33. 2021ClawCraneNet: Leveraging Object-level Relation for Text-based Video Segmentationcited by 21
  34. 2022Phrase-Based Affordance Detection via Cyclic Bilateral Interactioncited by 19
  35. 2023Referring Camouflaged Object Detectioncited by 19
  36. 2023Extending CLIP's Image-Text Alignment to Referring Image Segmentationcited by 19
  37. 2022Modeling Motion with Multi-Modal Features for Text-Based Video Segmentationcited by 18
  38. 2023Meta Compositional Referring Expression Segmentationcited by 18

50 most cited of 125 All 125 (JSON)