# MDC-R: The Minecraft Dialogue Corpus with Reference

arXiv: https://arxiv.org/abs/2506.22062 · HTML: https://mlsanity.com/paper/2506.22062/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 57 references that are in the map.
- First posted on arXiv 27 Jun 2025.

## Builds on (most cited first, 24 of 24)

- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning](https://mlsanity.com/paper/1612.06890/) (2016)
- [Generation and Comprehension of Unambiguous Object Descriptions](https://mlsanity.com/paper/1511.02283/) (2015)
- [AI2-THOR: An Interactive 3D Environment for Visual AI](https://mlsanity.com/paper/1712.05474/) (2017)
- [MDETR -- Modulated Detection for End-to-End Multi-Modal Understanding](https://mlsanity.com/paper/2104.12763/) (2021)
- [ALFRED: A Benchmark for Interpreting Grounded Instructions for Everyday Tasks](https://mlsanity.com/paper/1912.01734/) (2019)
- [VirtualHome: Simulating Household Activities via Programs](https://mlsanity.com/paper/1806.07011/) (2018)
- [REVERIE: Remote Embodied Visual Referring Expression in Real Indoor Environments](https://mlsanity.com/paper/1904.10151/) (2019)
- [Modeling Context Between Objects for Referring Expression Understanding](https://mlsanity.com/paper/1608.00525/) (2016)
- [Learning to Speak and Act in a Fantasy Text Adventure Game](https://mlsanity.com/paper/1903.03094/) (2019)
- [GuessWhat?! Visual object discovery through multi-modal dialogue](https://mlsanity.com/paper/1611.08481/) (2016)
- [Executing Instructions in Situated Collaborative Interactions](https://mlsanity.com/paper/1910.03655/) (2019)
- [MindCraft: Theory of Mind Modeling for Situated Dialogue in Collaborative Tasks](https://mlsanity.com/paper/2109.06275/) (2021)
- [Interactive Grounded Language Understanding in a Collaborative Environment: IGLU 2021](https://mlsanity.com/paper/2205.02388/) (2022)
- [What You See is What You Get: Visual Pronoun Coreference Resolution in Dialogues](https://mlsanity.com/paper/1909.00421/) (2019)
- [Advancing Visual Grounding with Scene Knowledge: Benchmark and Method](https://mlsanity.com/paper/2307.11558/) (2023)
- [Why Build an Assistant in Minecraft?](https://mlsanity.com/paper/1907.09273/) (2019)
- [Dialogue Games for Benchmarking Language Understanding: Motivation, Taxonomy, Strategy](https://mlsanity.com/paper/2304.07007/) (2023)
- [Give Me Something to Eat: Referring Expression Comprehension with Commonsense Knowledge](https://mlsanity.com/paper/2006.01629/) (2020)
- [VD-PCR: Improving Visual Dialog with Pronoun Coreference Resolution](https://mlsanity.com/paper/2205.14693/) (2022)
- [Who are you referring to? Coreference resolution in image narrations](https://mlsanity.com/paper/2211.14563/) (2022)
- [Find Someone Who: Visual Commonsense Understanding in Human-Centric Grounding](https://mlsanity.com/paper/2212.06971/) (2022)
- [Semi-supervised multimodal coreference resolution in image narrations](https://mlsanity.com/paper/2310.13619/) (2023)

## Built on by (most cited first, 2 of 2)

- [Referential ambiguity and clarification requests: comparing human and LLM behaviour](https://mlsanity.com/paper/2507.10445/) (2025)
- [Making Dialogue Grounding Data Rich: A Three-Tier Data Synthesis Framework for Generalized Referring Expression Comprehension](https://mlsanity.com/paper/2512.02791/) (2025)

## Abstract (arXiv)

We introduce the Minecraft Dialogue Corpus with Reference (MDC-R). MDC-R is a new language resource that supplements the original Minecraft Dialogue Corpus (MDC) with expert annotations of anaphoric and deictic reference. MDC's task-orientated, multi-turn, situated dialogue in a dynamic environment has motivated multiple annotation efforts, owing to the interesting linguistic phenomena that this setting gives rise to. We believe it can serve as a valuable resource when annotated with reference, too. Here, we discuss our method of annotation and the resulting corpus, and provide both a quantitative and a qualitative analysis of the data. Furthermore, we carry out a short experiment demonstrating the usefulness of our corpus for referring expression comprehension.