# MeetUp! A Corpus of Joint Activity Dialogues in a Visual Environment

arXiv: https://arxiv.org/abs/1907.05084 · HTML: https://mlsanity.com/paper/1907.05084/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jul 2019.

## Built on by (most cited first, 19 of 19)

- [Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directions](https://mlsanity.com/paper/2203.12667/) (2022)
- [Learning Rewards from Linguistic Feedback](https://mlsanity.com/paper/2009.14715/) (2020)
- [Effect of Visual Extensions on Natural Language Understanding in Vision-and-Language Models](https://mlsanity.com/paper/2104.08066/) (2021)
- [Reference-Centric Models for Grounded Collaborative Dialogue](https://mlsanity.com/paper/2109.05042/) (2021)
- [Pragmatics in Language Grounding: Phenomena, Tasks, and Modeling Approaches](https://mlsanity.com/paper/2211.08371/) (2022)
- [Dialogue Games for Benchmarking Language Understanding: Motivation, Taxonomy, Strategy](https://mlsanity.com/paper/2304.07007/) (2023)
- [Dealing with Semantic Underspecification in Multimodal NLP](https://mlsanity.com/paper/2306.05240/) (2023)
- [Conversational Grounding: Annotation and Analysis of Grounding Acts and Grounding Units](https://mlsanity.com/paper/2403.16609/) (2024)
- [What A Situated Language-Using Agent Must be Able to Do: A Top-Down Analysis](https://mlsanity.com/paper/2302.08590/) (2023)
- [The slurk Interaction Server Framework: Better Data for Better Dialog Models](https://mlsanity.com/paper/2202.01155/) (2022)
- [Resolving References in Visually-Grounded Dialogue via Text Generation](https://mlsanity.com/paper/2309.13430/) (2023)
- [Understanding Common Ground Misalignment in Goal-Oriented Dialog: A Case-Study with Ubuntu Chat Logs](https://mlsanity.com/paper/2503.12370/) (2025)
- [Using Game Play to Investigate Multimodal and Conversational Grounding in Large Multimodal Models](https://mlsanity.com/paper/2406.14035/) (2024)
- [A Third Paradigm for LLM Evaluation: Dialogue Game-Based Evaluation using clembench](https://mlsanity.com/paper/2507.08491/) (2025)
- [Frame of Reference: Addressing the Challenges of Common Ground Representation in Situational Dialogs](https://mlsanity.com/paper/2601.09365/) (2026)
- [Collecting Visually-Grounded Dialogue with A Game Of Sorts](https://mlsanity.com/paper/2309.05162/) (2023)
- [We went to look for meaning and all we got were these lousy representations: aspects of meaning representation for computational semantics](https://mlsanity.com/paper/2109.04949/) (2021)
- [Spot the Difference: A Cooperative Object-Referring Game in Non-Perfectly Co-Observable Scene](https://mlsanity.com/paper/2203.08362/) (2022)
- [Using Machine Mental Imagery for Representing Common Ground in Situated Dialogue](https://mlsanity.com/paper/2604.21144/) (2026)

## Abstract (arXiv)

Building computer systems that can converse about their visual environment is one of the oldest concerns of research in Artificial Intelligence and Computational Linguistics (see, for example, Winograd's 1972 SHRDLU system). Only recently, however, have methods from computer vision and natural language processing become powerful enough to make this vision seem more attainable. Pushed especially by developments in computer vision, many data sets and collection environments have recently been published that bring together verbal interaction and visual processing. Here, we argue that these datasets tend to oversimplify the dialogue part, and we propose a task---MeetUp!---that requires both visual and conversational grounding, and that makes stronger demands on representations of the discourse. MeetUp! is a two-player coordination game where players move in a visual environment, with the objective of finding each other. To do so, they must talk about what they see, and achieve mutual understanding. We describe a data collection and show that the resulting dialogues indeed exhibit the dialogue phenomena of interest, while also challenging the language & vision aspect.