# Refer, Reuse, Reduce: Generating Subsequent References in Visual and Conversational Contexts

arXiv: https://arxiv.org/abs/2011.04554 · HTML: https://mlsanity.com/paper/2011.04554/

## Key facts

- Cited by 22 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Nov 2020.

## Built on by (most cited first, 22 of 22)

- [Recent Advances in Deep Learning Based Dialogue Systems: A Systematic Survey](https://mlsanity.com/paper/2105.04387/) (2021)
- [Grounding 'Grounding' in NLP](https://mlsanity.com/paper/2106.02192/) (2021)
- [Pragmatics in the Era of Large Language Models: A Survey on Datasets, Evaluation, Opportunities and Challenges](https://mlsanity.com/paper/2502.12378/) (2025)
- [From partners to populations: A hierarchical Bayesian account of coordination and convention](https://mlsanity.com/paper/2104.05857/) (2021)
- [Reference-Centric Models for Grounded Collaborative Dialogue](https://mlsanity.com/paper/2109.05042/) (2021)
- [CoLLIE: Continual Learning of Language Grounding from Language-Image Embeddings](https://mlsanity.com/paper/2111.07993/) (2021)
- [Maintaining Common Ground in Dynamic Environments](https://mlsanity.com/paper/2105.14207/) (2021)
- [MUG: Interactive Multimodal Grounding on User Interfaces](https://mlsanity.com/paper/2209.15099/) (2022)
- [Dealing with Semantic Underspecification in Multimodal NLP](https://mlsanity.com/paper/2306.05240/) (2023)
- [Measuring How (Not Just Whether) VLMs Build Common Ground](https://mlsanity.com/paper/2509.03805/) (2025)
- [Speaking the Language of Your Listener: Audience-Aware Adaptation via Plug-and-Play Theory of Mind](https://mlsanity.com/paper/2305.19933/) (2023)
- [Post-training for Efficient Communication via Convention Formation](https://mlsanity.com/paper/2508.06482/) (2025)
- [Success and Cost Elicit Convention Formation for Efficient Communication](https://mlsanity.com/paper/2510.24023/) (2025)
- [When Personalization Legitimizes Risks: Uncovering Safety Vulnerabilities in Personalized Dialogue Agents](https://mlsanity.com/paper/2601.17887/) (2026)
- [Resolving References in Visually-Grounded Dialogue via Text Generation](https://mlsanity.com/paper/2309.13430/) (2023)
- [I see what you mean: Co-Speech Gestures for Reference Resolution in Multimodal Dialogue](https://mlsanity.com/paper/2503.00071/) (2025)
- [Referring Expression Generation in Visually Grounded Dialogue with Discourse-aware Comprehension Guiding](https://mlsanity.com/paper/2409.05721/) (2024)
- [Investigating Multimodal Informativity under Different Partner Visibility Conditions in Video-Mediated Dialogue](https://mlsanity.com/paper/2608.08915/) (2026)
- [Spot the Difference: A Cooperative Object-Referring Game in Non-Perfectly Co-Observable Scene](https://mlsanity.com/paper/2203.08362/) (2022)
- [Listener Model for the PhotoBook Referential Game with CLIPScores as Implicit Reference Chain](https://mlsanity.com/paper/2306.09607/) (2023)
- [Ad hoc conventions generalize to new referents](https://mlsanity.com/paper/2509.05566/) (2025)
- [Which one is banana man? Evaluating vision-language models in multi-turn pragmatic interpretation](https://mlsanity.com/paper/2608.29571/) (2026)

## Abstract (arXiv)

Dialogue participants often refer to entities or situations repeatedly within a conversation, which contributes to its cohesiveness. Subsequent references exploit the common ground accumulated by the interlocutors and hence have several interesting properties, namely, they tend to be shorter and reuse expressions that were effective in previous mentions. In this paper, we tackle the generation of first and subsequent references in visually grounded dialogue. We propose a generation model that produces referring utterances grounded in both the visual and the conversational context. To assess the referring effectiveness of its output, we also implement a reference resolution system. Our experiments and analyses show that the model produces better, more effective referring utterances than a model not grounded in the dialogue context, and generates subsequent references that exhibit linguistic patterns akin to humans.