# Know What You Don't Know: Modeling a Pragmatic Speaker that Refers to Objects of Unknown Categories

arXiv: https://arxiv.org/abs/1906.05518 · HTML: https://mlsanity.com/paper/1906.05518/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Jun 2019.

## Built on by (most cited first, 8 of 8)

- [Perspective-taking and Pragmatics for Generating Empathetic Responses Focused on Emotion Causes](https://mlsanity.com/paper/2109.08828/) (2021)
- [Learning to Mediate Disparities Towards Pragmatic Communication](https://mlsanity.com/paper/2203.13685/) (2022)
- [RSA-Control: A Pragmatics-Grounded Lightweight Controllable Text Generation Framework](https://mlsanity.com/paper/2410.19109/) (2024)
- [Speaking the Language of Your Listener: Audience-Aware Adaptation via Plug-and-Play Theory of Mind](https://mlsanity.com/paper/2305.19933/) (2023)
- [Cognitive Modeling with Scaffolded LLMs: A Case Study of Referential Expression Generation](https://mlsanity.com/paper/2407.03805/) (2024)
- [NeLLCom-Lex: A Neural-agent Framework to Study the Interplay between Lexical Systems and Language Use](https://mlsanity.com/paper/2509.22479/) (2025)
- [When in Doubt: Improving Classification Performance with Alternating Normalization](https://mlsanity.com/paper/2109.13449/) (2021)
- [Flout at Your Own Risk: LLMs Struggle with Pragmatic Cooperativity Under Epistemic Asymmetry](https://mlsanity.com/paper/2607.11053/) (2026)

## Abstract (arXiv)

Zero-shot learning in Language & Vision is the task of correctly labelling (or naming) objects of novel categories. Another strand of work in L&V aims at pragmatically informative rather than ``correct'' object descriptions, e.g. in reference games. We combine these lines of research and model zero-shot reference games, where a speaker needs to successfully refer to a novel object in an image. Inspired by models of "rational speech acts", we extend a neural generator to become a pragmatic speaker reasoning about uncertain object categories. As a result of this reasoning, the generator produces fewer nouns and names of distractor categories as compared to a literal speaker. We show that this conversational strategy for dealing with novel objects often improves communicative success, in terms of resolution accuracy of an automatic listener.