mlsanityData to 7 Oct 2026

Referring Expression Comprehension: A Survey of Methods and Datasets

arXiv 2007.09554 ↗First posted 19 Jul 2020cs.CV cs.CLcited by 78

Not on the map. Why?

Abstract (arXiv)

Referring expression comprehension (REC) aims to localize a target object in an image described by a referring expression phrased in natural language. Different from the object detection task that queried object labels have been pre-defined, the REC problem only can observe the queries during the test. It thus more challenging than a conventional computer vision problem. This task has attracted a lot of attention from both computer vision and natural language processing community, and several lines of work have been proposed, from CNN-RNN model, modular network to complex graph-based model. In this survey, we first examine the state of the art by comparing modern approaches to the problem. We classify methods by their mechanism to encode the visual and textual modalities. In particular, we examine the common approach of joint embedding images and expressions to a common feature space. We also discuss modular architectures and graph-based models that interface with structured graph representation. In the second part of this survey, we review the datasets available for training and evaluating REC systems. We then group results according to the datasets, backbone models, settings so that they can be fairly compared. Finally, we discuss promising future directions for the field, in particular the compositional referring expression comprehension that requires longer reasoning chain to address.

Key facts. Cited by 78 papers in the map (data through 7 Oct 2026). First posted on arXiv 19 Jul 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 7; 2022: 8; 2023: 10; 2024: 21; 2025: 17; 2026: 15.

78 papers.

Sort:

By area:

  1. 2023Alpha-CLIP: A CLIP Model Focusing on Wherever You Wantcited by 150
  2. 2024VisualAgentBench: Towards Large Multimodal Models as Visual Foundation Agentscited by 101
  3. 2024VL-Mamba: Exploring State Space Models for Multimodal Learningcited by 93
  4. 2025UniVG-R1: Reasoning Guided Universal Visual Grounding with Reinforcement Learningcited by 83
  5. 2021Free-form Description Guided 3D Visual Graph Network for Object Grounding in Point Cloudcited by 81
  6. 2024MyVLM: Personalizing VLMs for User-Specific Queriescited by 58
  7. 2024Toward Visual Grounding: A Surveycited by 52
  8. 2021Visual Grounding with Transformerscited by 42
  9. 2023CLIP-VG: Self-paced Curriculum Adapting of CLIP for Visual Groundingcited by 35
  10. 2024LocCa: Visual Pretraining with Location-aware Captionerscited by 35
  11. 2021INVIGORATE: Interactive Visual Grounding and Grasping in Cluttercited by 32
  12. 2024HiVG: Hierarchical Multimodal Fine-grained Modulation for Visual Groundingcited by 31
38 more
  1. 2024OneRef: Unified One-tower Expression Grounding and Segmentation with Mask Referring Modelingcited by 28
  2. 2023Multimodality Representation Learning: A Survey on Evolution, Pretraining and Its Applicationscited by 27
  3. 2024MC-GPT: Empowering Vision-and-Language Navigation with Memory Map and Reasoning Chainscited by 25
  4. 2024A Survey on Text-guided 3D Visual Grounding: Elements, Recent Advances, and Future Directionscited by 24
  5. 2023Language-guided Robot Grasping: CLIP-based Referring Grasp Synthesis in Cluttercited by 22
  6. 2025Multimodal Referring Segmentation: A Surveycited by 22
  7. 2025Referring to Any Personcited by 20
  8. 2021Who's Waldo? Linking People Across Text and Imagescited by 16
  9. 2023Dense Object Grounding in 3D Scenescited by 15
  10. 2022FindIt: Generalized Localization with Natural Language Queriescited by 13
  11. 2024WaterVG: Waterway Visual Grounding based on Text-Guided Vision and mmWave Radarcited by 13
  12. 2022Answer-Me: Multi-Task Open-Vocabulary Visual Question Answeringcited by 12
  13. 2023ScanERU: Interactive 3D Visual Grounding based on Embodied Reference Understandingcited by 12
  14. 2021CoLLIE: Continual Learning of Language Grounding from Language-Image Embeddingscited by 11
  15. 2023Toloka Visual Question Answering Benchmarkcited by 10
  16. 2021Evaluation of Audio-Visual Alignments in Visually Grounded Speech Modelscited by 8
  17. 2025SwimVG: Step-wise Multimodal Fusion and Adaption for Visual Groundingcited by 8
  18. 2025Grounding-Aware Token Pruning: Recovering from Drastic Performance Drops in Visual Grounding Caused by Pruningcited by 8
  19. 2025VG-Refiner: Towards Tool-Refined Referring Grounded Reasoning via Agentic Reinforcement Learningcited by 8
  20. 2023Multimodal Diffusion Segmentation Model for Object Segmentation from Manipulation Instructionscited by 7
  21. 2024NanoMVG: USV-Centric Low-Power Multi-Task Visual Grounding based on Prompt-Guided Camera and 4D mmWave Radarcited by 7
  22. 2025Acknowledging Focus Ambiguity in Visual Questionscited by 7
  23. 2025GroundingME: Exposing the Visual Grounding Gap in MLLMs through Multi-Dimensional Evaluationcited by 7
  24. 2024J-CRe3: A Japanese Conversation Dataset for Real-world Reference Resolutioncited by 6
  25. 2024Temporal-Enhanced Multimodal Transformer for Referring Multi-Object Tracking and Segmentationcited by 6
  26. 2025GeoRSMLLM: A Multimodal Large Language Model for Vision-Language Tasks in Geoscience and Remote Sensingcited by 6
  27. 2022A Survivor in the Era of Large-Scale Pretraining: An Empirical Study of One-Stage Referring Expression Comprehensioncited by 5
  28. 2024Learn from Foundation Model: Fruit Detection Model without Manual Annotationcited by 5
  29. 2022Suspected Object Matters: Rethinking Model's Prediction for One-stage Visual Groundingcited by 4
  30. 2022Find Someone Who: Visual Commonsense Understanding in Human-Centric Groundingcited by 4
  31. 2024UEMM-Air: Make Unmanned Aerial Vehicles Perform More Multi-modal Taskscited by 4
  32. 2026DVGBench: Implicit-to-Explicit Visual Grounding Benchmark in UAV Imagery with Large Vision-Language Modelscited by 4
  33. 2022Towards Automated Error Analysis: Learning to Characterize Errorscited by 3
  34. 2024LLM-wrapper: Black-Box Semantic-Aware Adaptation of Vision-Language Models for Referring Expression Comprehensioncited by 3
  35. 20263D Instruction Ambiguity Detectioncited by 3
  36. 2022To Find Waldo You Need Contextual Cues: Debiasing Who's Waldocited by 2
  37. 2022Extending Phrase Grounding with Pronouns in Visual Dialoguescited by 2
  38. 2023Natural Language Robot Programming: NLP integrated with autonomous robotic graspingcited by 2

50 most cited of 78 All 78 (JSON)