# GREC: Generalized Referring Expression Comprehension

arXiv: https://arxiv.org/abs/2308.16182 · HTML: https://mlsanity.com/paper/2308.16182/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 41 references that are in the map.
- First posted on arXiv 30 Aug 2023.

## Builds on (most cited first, 30 of 35)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection](https://mlsanity.com/paper/2303.05499/) (2023)
- [Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotations](https://mlsanity.com/paper/1602.07332/) (2016)
- [Modeling Context in Referring Expressions](https://mlsanity.com/paper/1608.00272/) (2016)
- [Generation and Comprehension of Unambiguous Object Descriptions](https://mlsanity.com/paper/1511.02283/) (2015)
- [MDETR -- Modulated Detection for End-to-End Multi-Modal Understanding](https://mlsanity.com/paper/2104.12763/) (2021)
- [TransVG: End-to-End Visual Grounding with Transformers](https://mlsanity.com/paper/2104.08541/) (2021)
- [GRES: Generalized Referring Expression Segmentation](https://mlsanity.com/paper/2306.00968/) (2023)
- [Vision-Language Transformer and Query Generation for Referring Segmentation](https://mlsanity.com/paper/2108.05565/) (2021)
- [MOSE: A New Dataset for Video Object Segmentation in Complex Scenes](https://mlsanity.com/paper/2302.01872/) (2023)
- [MeViS: A Large-scale Benchmark for Video Segmentation with Motion Expressions](https://mlsanity.com/paper/2308.08544/) (2023)
- [Multi-task Collaborative Network for Joint Referring Expression Comprehension and Segmentation](https://mlsanity.com/paper/2003.08813/) (2020)
- [A Fast and Accurate One-Stage Approach to Visual Grounding](https://mlsanity.com/paper/1908.06354/) (2019)
- [Universal Instance Perception as Object Discovery and Retrieval](https://mlsanity.com/paper/2303.06674/) (2023)
- [Towards Open Vocabulary Learning: A Survey](https://mlsanity.com/paper/2306.15880/) (2023)
- [Learning to Assemble Neural Module Tree Networks for Visual Grounding](https://mlsanity.com/paper/1812.03299/) (2018)
- [Improving One-stage Visual Grounding by Recursive Sub-query Construction](https://mlsanity.com/paper/2008.01059/) (2020)
- [Natural Language Object Retrieval](https://mlsanity.com/paper/1511.04164/) (2015)
- [Transformer-Based Visual Segmentation: A Survey](https://mlsanity.com/paper/2304.09854/) (2023)
- [PhraseCut: Language-based Image Segmentation in the Wild](https://mlsanity.com/paper/2008.01187/) (2020)
- [A Real-Time Cross-modality Correlation Filtering Method for Referring Expression Comprehension](https://mlsanity.com/paper/1909.07072/) (2019)
- [VLT: Vision-Language Transformer and Query Generation for Referring Segmentation](https://mlsanity.com/paper/2210.15871/) (2022)
- [Modeling Relationships in Referential Expressions with Compositional Modular Networks](https://mlsanity.com/paper/1611.09978/) (2016)
- [Neighbourhood Watch: Referring Expression Comprehension via Language-guided Graph Attention Networks](https://mlsanity.com/paper/1812.04794/) (2018)
- [Learning to Compose and Reason with Language Tree Structures for Visual Grounding](https://mlsanity.com/paper/1906.01784/) (2019)
- [Grounding Referring Expressions in Images by Variational Context](https://mlsanity.com/paper/1712.01892/) (2017)
- [Parallel Attention: A Unified Framework for Visual Object Discovery through Dialogs and Queries](https://mlsanity.com/paper/1711.06370/) (2017)
- [Real-Time Referring Expression Comprehension by Single-Stage Grounding Network](https://mlsanity.com/paper/1812.03426/) (2018)
- [Towards Robust Referring Image Segmentation](https://mlsanity.com/paper/2209.09554/) (2022)
- [Instance-Specific Feature Propagation for Referring Segmentation](https://mlsanity.com/paper/2204.12109/) (2022)

## Built on by (most cited first, 30 of 42)

- [Your Large Vision-Language Model Only Needs A Few Attention Heads For Visual Grounding](https://mlsanity.com/paper/2503.06287/) (2025)
- [Vision-R1: Evolving Human-Free Alignment in Large Vision-Language Models via Vision-Guided Reinforcement Learning](https://mlsanity.com/paper/2503.18013/) (2025)
- [Jack of All Tasks, Master of Many: Designing General-purpose Coarse-to-Fine Vision-Language Model](https://mlsanity.com/paper/2312.12423/) (2023)
- [Toward Visual Grounding: A Survey](https://mlsanity.com/paper/2412.20206/) (2024)
- [VoCoT: Unleashing Visually Grounded Multi-Step Reasoning in Large Multi-Modal Models](https://mlsanity.com/paper/2405.16919/) (2024)
- [SimVG: A Simple Framework for Visual Grounding with Decoupled Multi-modal Fusion](https://mlsanity.com/paper/2409.17531/) (2024)
- [Multimodal Referring Segmentation: A Survey](https://mlsanity.com/paper/2508.00265/) (2025)
- [Geo-R1: Improving Few-Shot Geospatial Referring Expression Understanding with Reinforcement Fine-Tuning](https://mlsanity.com/paper/2509.21976/) (2025)
- [InstructDET: Diversifying Referring Object Detection with Generalized Instructions](https://mlsanity.com/paper/2310.05136/) (2023)
- [DeRIS: Decoupling Perception and Cognition for Enhanced Referring Image Segmentation through Loopback Synergy](https://mlsanity.com/paper/2507.01738/) (2025)
- [RefChartQA: Grounding Visual Answer on Chart Images through Instruction Tuning](https://mlsanity.com/paper/2503.23131/) (2025)
- [PropVG: End-to-End Proposal-Driven Visual Grounding with Multi-Granularity Discrimination](https://mlsanity.com/paper/2509.04833/) (2025)
- [Towards Understanding Visual Grounding in Visual Language Models](https://mlsanity.com/paper/2509.10345/) (2025)
- [TACO: Think-Answer Consistency for Optimized Long-Chain Reasoning and Efficient Data Learning via Reinforcement Learning in LVLMs](https://mlsanity.com/paper/2505.20777/) (2025)
- [ActionVOS: Actions as Prompts for Video Object Segmentation](https://mlsanity.com/paper/2407.07402/) (2024)
- [LSVOS Challenge Report: Large-scale Complex and Long Video Object Segmentation](https://mlsanity.com/paper/2409.05847/) (2024)
- [Improving Generalized Visual Grounding with Instance-aware Joint Learning](https://mlsanity.com/paper/2509.13747/) (2025)
- [Beyond Unimodal Shortcuts: MLLMs as Cross-Modal Reasoners for Grounded Named Entity Recognition](https://mlsanity.com/paper/2602.04486/) (2026)
- [InstructSAM: Segment Any Instance with Any Instructions](https://mlsanity.com/paper/2605.26102/) (2026)
- [PVUW 2025 Challenge Report: Advances in Pixel-level Understanding of Complex Videos in the Wild](https://mlsanity.com/paper/2504.11326/) (2025)
- [LSVOS 2025 Challenge Report: Recent Advances in Complex Video Object Segmentation](https://mlsanity.com/paper/2510.11063/) (2025)
- [Griffon-G: Bridging Vision-Language and Vision-Centric Tasks via Large Multimodal Models](https://mlsanity.com/paper/2410.16163/) (2024)
- [Understand, Think, and Answer: Advancing Visual Reasoning with Large Multimodal Models](https://mlsanity.com/paper/2505.20753/) (2025)
- [Multi-label Instance-level Generalised Visual Grounding in Agriculture](https://mlsanity.com/paper/2603.06699/) (2026)
- [RefHCM: A Unified Model for Referring Perceptions in Human-Centric Scenarios](https://mlsanity.com/paper/2412.14643/) (2024)
- [Synthetic Object Compositions for Scalable and Accurate Learning in Detection, Segmentation, and Grounding](https://mlsanity.com/paper/2510.09110/) (2025)
- [Comprehension of Multilingual Expressions Referring to Target Objects in Visual Inputs](https://mlsanity.com/paper/2511.11427/) (2025)
- [LIHE: Linguistic Instance-Split Hyperbolic-Euclidean Framework for Generalized Weakly-Supervised Referring Expression Comprehension](https://mlsanity.com/paper/2511.12020/) (2025)
- [Generalised Medical Phrase Grounding](https://mlsanity.com/paper/2512.01085/) (2025)
- [VIRO: Robust and Efficient Neuro-Symbolic Reasoning with Verification for Referring Expression Comprehension](https://mlsanity.com/paper/2601.12781/) (2026)

## Abstract (arXiv)

The objective of Classic Referring Expression Comprehension (REC) is to produce a bounding box corresponding to the object mentioned in a given textual description. Commonly, existing datasets and techniques in classic REC are tailored for expressions that pertain to a single target, meaning a sole expression is linked to one specific object. Expressions that refer to multiple targets or involve no specific target have not been taken into account. This constraint hinders the practical applicability of REC. This study introduces a new benchmark termed as Generalized Referring Expression Comprehension (GREC). This benchmark extends the classic REC by permitting expressions to describe any number of target objects. To achieve this goal, we have built the first large-scale GREC dataset named gRefCOCO. This dataset encompasses a range of expressions: those referring to multiple targets, expressions with no specific target, and the single-target expressions. The design of GREC and gRefCOCO ensures smooth compatibility with classic REC. The proposed gRefCOCO dataset, a GREC method implementation code, and GREC evaluation code are available at https://github.com/henghuiding/gRefCOCO.