# Pragmatic inference and visual abstraction enable contextual flexibility during visual communication

arXiv: https://arxiv.org/abs/1903.04448 · HTML: https://mlsanity.com/paper/1903.04448/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Mar 2019.

## Built on by (most cited first, 14 of 14)

- [CLIPasso: Semantically-Aware Object Sketching](https://mlsanity.com/paper/2202.05822/) (2022)
- [SketchAgent: Language-Driven Sequential Sketch Generation](https://mlsanity.com/paper/2411.17673/) (2024)
- [From partners to populations: A hierarchical Bayesian account of coordination and convention](https://mlsanity.com/paper/2104.05857/) (2021)
- [Learning to Draw: Emergent Communication through Sketching](https://mlsanity.com/paper/2106.02067/) (2021)
- [Emergent Graphical Conventions in a Visual Communication Game](https://mlsanity.com/paper/2111.14210/) (2021)
- [SEVA: Leveraging sketches to evaluate alignment between human and machine visual abstraction](https://mlsanity.com/paper/2312.03035/) (2023)
- [Learning with Language-Guided State Abstractions](https://mlsanity.com/paper/2402.18759/) (2024)
- [Adaptive Language-Guided Abstraction from Contrastive Explanations](https://mlsanity.com/paper/2409.08212/) (2024)
- [Visual communication of object concepts at different levels of abstraction](https://mlsanity.com/paper/2106.02775/) (2021)
- [Emergent Communication in Interactive Sketch Question Answering](https://mlsanity.com/paper/2310.15597/) (2023)
- [Deep Sketch-Based 3D Modeling: A Survey](https://mlsanity.com/paper/2603.03287/) (2026)
- [Visual resemblance and communicative context constrain the emergence of graphical conventions](https://mlsanity.com/paper/2109.13861/) (2021)
- [Visual Theory of Mind Enables the Invention of Proto-Writing](https://mlsanity.com/paper/2502.01568/) (2025)
- [Using Machine Mental Imagery for Representing Common Ground in Situated Dialogue](https://mlsanity.com/paper/2604.21144/) (2026)

## Abstract (arXiv)

Visual modes of communication are ubiquitous in modern life --- from maps to data plots to political cartoons. Here we investigate drawing, the most basic form of visual communication. Participants were paired in an online environment to play a drawing-based reference game. On each trial, both participants were shown the same four objects, but in different locations. The sketcher's goal was to draw one of these objects so that the viewer could select it from the array. On `close' trials, objects belonged to the same basic-level category, whereas on `far' trials objects belonged to different categories. We found that people exploited shared information to efficiently communicate about the target object: on far trials, sketchers achieved high recognition accuracy while applying fewer strokes, using less ink, and spending less time on their drawings than on close trials. We hypothesized that humans succeed in this task by recruiting two core faculties: visual abstraction, the ability to perceive the correspondence between an object and a drawing of it; and pragmatic inference, the ability to judge what information would help a viewer distinguish the target from distractors. To evaluate this hypothesis, we developed a computational model of the sketcher that embodied both faculties, instantiated as a deep convolutional neural network nested within a probabilistic program. We found that this model fit human data well and outperformed lesioned variants. Together, this work provides the first algorithmically explicit theory of how visual perception and social cognition jointly support contextual flexibility in visual communication.