# Making Dialogue Grounding Data Rich: A Three-Tier Data Synthesis Framework for Generalized Referring Expression Comprehension

arXiv: https://arxiv.org/abs/2512.02791 · HTML: https://mlsanity.com/paper/2512.02791/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 27 references that are in the map.
- First posted on arXiv 2 Dec 2025.

## Builds on (most cited first, 18 of 18)

- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Models](https://mlsanity.com/paper/1505.04870/) (2015)
- [Modeling Context in Referring Expressions](https://mlsanity.com/paper/1608.00272/) (2016)
- [MDETR -- Modulated Detection for End-to-End Multi-Modal Understanding](https://mlsanity.com/paper/2104.12763/) (2021)
- [ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks](https://mlsanity.com/paper/1912.01734/) (2019)
- [MetaMath: Bootstrap Your Own Mathematical Questions for Large Language Models](https://mlsanity.com/paper/2309.12284/) (2023)
- [MAttNet: Modular Attention Network for Referring Expression Comprehension](https://mlsanity.com/paper/1801.08186/) (2018)
- [CLEVR-Ref+: Diagnosing Visual Reasoning with Referring Expressions](https://mlsanity.com/paper/1901.00850/) (2019)
- [MathGenie: Generating Synthetic Data with Question Back-translation for Enhancing Mathematical Reasoning of LLMs](https://mlsanity.com/paper/2402.16352/) (2024)
- [Jack of All Tasks, Master of Many: Designing General-purpose Coarse-to-Fine Vision-Language Model](https://mlsanity.com/paper/2312.12423/) (2023)
- [Cops-Ref: A new Dataset and Task on Compositional Referring Expression Comprehension](https://mlsanity.com/paper/2003.00403/) (2020)
- [GREC: Generalized Referring Expression Comprehension](https://mlsanity.com/paper/2308.16182/) (2023)
- [Interactive Grounded Language Understanding in a Collaborative Environment: IGLU 2021](https://mlsanity.com/paper/2205.02388/) (2022)
- [What You See is What You Get: Visual Pronoun Coreference Resolution in Dialogues](https://mlsanity.com/paper/1909.00421/) (2019)
- [Advancing Visual Grounding with Scene Knowledge: Benchmark and Method](https://mlsanity.com/paper/2307.11558/) (2023)
- [Give Me Something to Eat: Referring Expression Comprehension with Commonsense Knowledge](https://mlsanity.com/paper/2006.01629/) (2020)
- [Harlequin: Color-driven Generation of Synthetic Data for Referring Expression Comprehension](https://mlsanity.com/paper/2411.14807/) (2024)
- [MDC-R: The Minecraft Dialogue Corpus with Reference](https://mlsanity.com/paper/2506.22062/) (2025)

## Abstract (arXiv)

Dialogue-Based Generalized Referring Expression Comprehension (GREC) requires models to ground the expression and unlimited targets in complex visual scenes while resolving coreference across a long dialogue context. However, existing systems struggle under distribution shift between training and evaluation domains, a gap exacerbated by the scarcity of annotated dialogue grounding data. We address this challenge with a three-tier data-synthesis method that balances realism and controllability to produce scalable supervision for dialogue-conditioned grounding. Fine-tuning on the synthesized data yields consistent, substantial improvements over prior approaches across standard evaluation metrics.