mlsanityData to 14 Oct 2026

Towards Complex-query Referring Image Segmentation: A Novel Benchmark

arXiv 2309.17205 ↗First posted 29 Sep 2023cs.CVcited by 10

Not on the map. Why?

Abstract (arXiv)

Referring Image Understanding (RIS) has been extensively studied over the past decade, leading to the development of advanced algorithms. However, there has been a lack of research investigating how existing algorithms should be benchmarked with complex language queries, which include more informative descriptions of surrounding objects and backgrounds (eg textit"the black car." vs. textit"the black car is parking on the road and beside the bus."). Given the significant improvement in the semantic understanding capability of large pre-trained models, it is crucial to take a step further in RIS by incorporating complex language that resembles real-world applications. To close this gap, building upon the existing RefCOCO and Visual Genome datasets, we propose a new RIS benchmark with complex queries, namely RIS-CQ. The RIS-CQ dataset is of high quality and large scale, which challenges the existing RIS with enriched, specific and informative queries, and enables a more realistic scenario of RIS research. Besides, we present a nichetargeting method to better task the RIS-CQ, called dual-modality graph alignment model (textbftextscDuMoGa), which outperforms a series of RIS methods.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 51 of its 65 references that are in the map. First posted on arXiv 29 Sep 2023.

Builds on

51 of 65 references.

References · 51

  1. 2017Attention Is All You Needcited by 60,361
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  3. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,139
  6. 2023Segment Anythingcited by 8,234
  7. 2017Mask R-CNNcited by 7,277
  8. 2014Fully Convolutional Networks for Semantic Segmentationcited by 4,859
  9. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  10. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  11. 2015VQA: Visual Question Answeringcited by 2,506
  12. 2018Deep contextualized word representationscited by 2,290
38 more
  1. 2016Modeling Context in Referring Expressionscited by 1,063
  2. 2015Generation and Comprehension of Unambiguous Object Descriptionscited by 904
  3. 2018Panoptic Segmentationcited by 802
  4. 2017Scene Graph Generation by Iterative Message Passingcited by 462
  5. 2018MAttNet: Modular Attention Network for Referring Expression Comprehensioncited by 431
  6. 2017Neural Motifs: Scene Graph Parsing with Global Contextcited by 414
  7. 2020Unbiased Scene Graph Generation from Biased Trainingcited by 356
  8. 2021LAVT: Language-Aware Vision Transformer for Referring Image Segmentationcited by 348
  9. 2018Reinforced Cross-Modal Matching and Self-Supervised Imitation Learning for Vision-Language Navigationcited by 305
  10. 2023Unleashing Text-to-Image Diffusion Models for Visual Perceptioncited by 269
  11. 2019Cross-Modal Self-Attention Network for Referring Image Segmentationcited by 263
  12. 2018Learning to Compose Dynamic Tree Structures for Visual Contextscited by 259
  13. 2016Segmentation from Natural Language Expressionscited by 257
  14. 2021Vision-Language Transformer and Query Generation for Referring Segmentationcited by 240
  15. 2020Multi-task Collaborative Network for Joint Referring Expression Comprehension and Segmentationcited by 204
  16. 2019A Fast and Accurate One-Stage Approach to Visual Groundingcited by 195
  17. 2023Universal Instance Perception as Object Discovery and Retrievalcited by 192
  18. 2020GPS-Net: Graph Property Sensing Network for Scene Graph Generationcited by 166
  19. 2019Knowledge-Embedded Routing Network for Scene Graph Generationcited by 164
  20. 2019ManiGAN: Text-Guided Image Manipulationcited by 156
  21. 2020Improving One-stage Visual Grounding by Recursive Sub-query Constructioncited by 138
  22. 2017Recurrent Multimodal Interaction for Referring Image Segmentationcited by 125
  23. 2016Graph-Structured Representations for Visual Question Answeringcited by 123
  24. 2019Dynamic Graph Attention for Referring Expression Comprehensioncited by 111
  25. 2021Encoder Fusion Network with Co-Attention Embedding for Referring Image Segmentationcited by 111
  26. 2021Locate then Segment: A Strong Pipeline for Referring Image Segmentationcited by 98
  27. 2019Multimodal Intelligence: Representation Learning, Information Fusion, and Applicationscited by 89
  28. 2020Say As You Wish: Fine-grained Control of Image Caption Generation with Abstract Scene Graphscited by 87
  29. 2018Dynamic Multimodal Instance Segmentation guided by natural language queriescited by 83
  30. 2019Attention Guided Graph Convolutional Networks for Relation Extractioncited by 82
  31. 2019Zero-Shot Grounding of Objects from Natural Language Queriescited by 79
  32. 2020CogTree: Cognition Tree Loss for Unbiased Scene Graph Generationcited by 62
  33. 2022Fine-Grained Scene Graph Generation with Data Transfercited by 57
  34. 2018Real-Time Referring Expression Comprehension by Single-Stage Grounding Networkcited by 56
  35. 2017Language-Based Image Editing with Recurrent Attentive Modelscited by 53
  36. 2020Scene Graph Reasoning for Visual Question Answeringcited by 50
  37. 2023Panoptic Scene Graph Generation with Semantics-Prototype Learningcited by 42
  38. 2021Word2Pix: Word to Pixel Cross Attention Transformer in Visual Groundingcited by 16

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 4; 2025: 4; 2026: 1.

10 papers.

  1. 2024Image Segmentation in Foundation Model Era: A Surveycited by 35
  2. 2025RefDrone: A Challenging Benchmark for Referring Expression Comprehension in Drone Scenescited by 19
  3. 2026FORTIS: Benchmarking Over-Privilege in Agent Skillscited by 13
  4. 2025Charts Are Not Images: On the Challenges of Scientific Chart Editingcited by 11
  5. 2024Context-Enhanced Video Moment Retrieval with Large Language Modelscited by 9
  6. 2023Domain-wise Invariant Learning for Panoptic Scene Graph Generationcited by 7
  7. 2024Backpropagation-Free Multi-modal On-Device Model Adaptation via Cloud-Device Collaborationcited by 5
  8. 2024Grounding is All You Need? Dual Temporal Grounding for Video Dialogcited by 1
  9. 2025Re-purposing SAM into Efficient Visual Projectors for MLLM-Based Referring Image Segmentationcited by 1
  10. 2025Planner-Refiner: Dynamic Space-Time Refinement for Vision-Language Alignment in Videoscited by 0