mlsanityData to 7 Oct 2026

Dynamic Multimodal Instance Segmentation guided by natural language queries

arXiv 1807.02257 ↗First posted 6 Jul 2018cs.CVcited by 83

Not on the map. Why?

Abstract (arXiv)

We address the problem of segmenting an object given a natural language expression that describes it. Current techniques tackle this task by either (textiti) directly or recursively merging linguistic and visual information in the channel dimension and then performing convolutions; or by (textitii) mapping the expression to a space in which it can be thought of as a filter, whose response is directly related to the presence of the object at a given spatial coordinate in the image, so that a convolution can be applied to look for the object. We propose a novel method that integrates these two insights in order to fully exploit the recursive nature of language. Additionally, during the upsampling process, we take advantage of the intermediate information generated when downsampling the image, so that detailed segmentations can be obtained. We compare our method against the state-of-the-art approaches in four standard datasets, in which it surpasses all previous methods in six of eight of the splits for this task.

Key facts. Cited by 83 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Jul 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 15; 2022: 15; 2023: 29; 2024: 13; 2025: 7; 2026: 4.

83 papers.

Sort:

By area:

  1. 2023Segment Everything Everywhere All at Oncecited by 546
  2. 2021CRIS: CLIP-Driven Referring Image Segmentationcited by 353
  3. 2021LAVT: Language-Aware Vision Transformer for Referring Image Segmentationcited by 347
  4. 2022Generalized Decoding for Pixel, Image, and Languagecited by 292
  5. 2023GRES: Generalized Referring Expression Segmentationcited by 252
  6. 2021Vision-Language Transformer and Query Generation for Referring Segmentationcited by 239
  7. 2023MeViS: A Large-scale Benchmark for Video Segmentation with Motion Expressionscited by 225
  8. 2023PolyFormer: Referring Image Segmentation as Sequential Polygon Generationcited by 147
  9. 2021Referring Transformer: A One-step Approach to Multi-task Visual Groundingcited by 143
  10. 2022ReSTR: Convolution-free Referring Image Segmentation Using Transformerscited by 112
  11. 2021Encoder Fusion Network with Co-Attention Embedding for Referring Image Segmentationcited by 111
  12. 2022VLT: Vision-Language Transformer and Query Generation for Referring Segmentationcited by 110
38 more
  1. 2021Cross-Modal Progressive Comprehension for Referring Segmentationcited by 102
  2. 2021Locate then Segment: A Strong Pipeline for Referring Image Segmentationcited by 98
  3. 2023Rotated Multi-Scale Interaction Network for Referring Remote Sensing Image Segmentationcited by 76
  4. 2023Beyond One-to-One: Rethinking the Referring Image Segmentationcited by 68
  5. 2023RRSIS: Referring Remote Sensing Image Segmentationcited by 59
  6. 2024Decoupling Static and Hierarchical Motion Perception for Referring Video Segmentationcited by 55
  7. 2023Referred by Multi-Modality: A Unified Temporal Transformer for Video Object Segmentationcited by 54
  8. 2023Contrastive Grouping with Transformer for Referring Image Segmentationcited by 54
  9. 2022Instance-Specific Feature Propagation for Referring Segmentationcited by 45
  10. 2022Local-Global Context Aware Transformer for Language-Guided Video Segmentationcited by 41
  11. 2024ReMamber: Referring Image Segmentation with Mamba Twistercited by 38
  12. 2023Multi-Modal Mutual Attention and Iterative Interaction for Referring Image Segmentationcited by 36
  13. 20233D-STMN: Dependency-Driven Superpoint-Text Matching Network for End-to-End 3D Referring Expression Segmentationcited by 36
  14. 2023Referring Image Segmentation Using Text Supervisioncited by 35
  15. 2022Peekaboo: Text to Image Diffusion Models are Zero-Shot Segmentorscited by 34
  16. 2023FoodLMM: A Versatile Food Assistant using Large Multi-modal Modelcited by 32
  17. 2023LD-ZNet: A Latent Diffusion Approach for Text-Based Image Segmentationcited by 29
  18. 2021Referring Segmentation in Images and Videos with Cross-Modal Self-Attention Networkcited by 28
  19. 2023Parallel Vertex Diffusion for Unified Visual Groundingcited by 28
  20. 2021Panoptic Narrative Groundingcited by 27
  21. 2023Shatter and Gather: Learning Referring Image Segmentation with Text Supervisioncited by 25
  22. 2024Exploring Fine-Grained Image-Text Alignment for Referring Remote Sensing Image Segmentationcited by 24
  23. 2024Curriculum Point Prompting for Weakly-Supervised Referring Image Segmentationcited by 23
  24. 2025Multimodal Referring Segmentation: A Surveycited by 22
  25. 2021ClawCraneNet: Leveraging Object-level Relation for Text-based Video Segmentationcited by 21
  26. 2022Phrase-Based Affordance Detection via Cyclic Bilateral Interactioncited by 19
  27. 2023Extending CLIP's Image-Text Alignment to Referring Image Segmentationcited by 19
  28. 2023Meta Compositional Referring Expression Segmentationcited by 18
  29. 2023Semantics-Aware Dynamic Localization and Refinement for Referring Image Segmentationcited by 16
  30. 2022Dynamic MLP for Fine-Grained Image Classification by Leveraging Geographical and Temporal Informationcited by 15
  31. 2023CM-MaskSD: Cross-Modality Masked Self-Distillation for Referring Image Segmentationcited by 15
  32. 2021MaIL: A Unified Mask-Image-Language Trimodal Network for Referring Image Segmentationcited by 14
  33. 2022Fully and Weakly Supervised Referring Expression Segmentation with End-to-End Learningcited by 14
  34. 2021Two-stage Visual Cues Enhancement Network for Referring Image Segmentationcited by 13
  35. 2022FindIt: Generalized Localization with Natural Language Queriescited by 13
  36. 2023Towards Real-Time Panoptic Narrative Grounding by an End-to-End Grounding Networkcited by 13
  37. 2024High-Quality Entity Segmentation and Groundingcited by 13
  38. 2022Answer-Me: Multi-Task Open-Vocabulary Visual Question Answeringcited by 12

50 most cited of 83 All 83 (JSON)