# Learning to Draw: Emergent Communication through Sketching

arXiv: https://arxiv.org/abs/2106.02067 · HTML: https://mlsanity.com/paper/2106.02067/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 50 references that are in the map.
- First posted on arXiv 3 Jun 2021.

## Builds on (most cited first, 23 of 23)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Barlow Twins: Self-Supervised Learning via Redundancy Reduction](https://mlsanity.com/paper/2103.03230/) (2021)
- [ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness](https://mlsanity.com/paper/1811.12231/) (2018)
- [Learning to Communicate with Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1605.06676/) (2016)
- [Emergence of Grounded Compositional Language in Multi-Agent Populations](https://mlsanity.com/paper/1703.04908/) (2017)
- [TarMAC: Targeted Multi-Agent Communication](https://mlsanity.com/paper/1810.11187/) (2018)
- [Multi-Agent Cooperation and the Emergence of (Natural) Language](https://mlsanity.com/paper/1612.07182/) (2016)
- [Emergence of Language with Multi-agent Games: Learning to Communicate with Sequences of Symbols](https://mlsanity.com/paper/1705.11192/) (2017)
- [Compositionality and Generalization in Emergent Languages](https://mlsanity.com/paper/2004.09124/) (2020)
- [Emergence of Linguistic Communication from Referential Games with Symbolic and Pixel Input](https://mlsanity.com/paper/1804.03984/) (2018)
- [Compositional Languages Emerge in a Neural Iterated Learning Model](https://mlsanity.com/paper/2002.01365/) (2020)
- [Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning](https://mlsanity.com/paper/1703.06585/) (2017)
- [On the Pitfalls of Measuring Emergent Communication](https://mlsanity.com/paper/1903.05168/) (2019)
- [Emergent Communication through Negotiation](https://mlsanity.com/paper/1804.03980/) (2018)
- [Anti-efficient encoding in emergent communication](https://mlsanity.com/paper/1905.12561/) (2019)
- [Differentiable Drawing and Sketching](https://mlsanity.com/paper/2103.16194/) (2021)
- [Learning to Play Guess Who? and Inventing a Grounded Language as a Consequence](https://mlsanity.com/paper/1611.03218/) (2016)
- [Pragmatic inference and visual abstraction enable contextual flexibility during visual communication](https://mlsanity.com/paper/1903.04448/) (2019)
- [Inductive Bias and Language Expressivity in Emergent Communication](https://mlsanity.com/paper/2012.02875/) (2020)
- [From Language Games to Drawing Games](https://mlsanity.com/paper/2010.02820/) (2020)
- [Emergence of Numeric Concepts in Multi-Agent Autonomous Communication](https://mlsanity.com/paper/1911.01098/) (2019)

## Built on by (most cited first, 23 of 23)

- [CLIPascene: Scene Sketching with Different Types and Levels of Abstraction](https://mlsanity.com/paper/2211.17256/) (2022)
- [CLIPasso: Semantically-Aware Object Sketching](https://mlsanity.com/paper/2202.05822/) (2022)
- [Breathing Life Into Sketches Using Text-to-Video Priors](https://mlsanity.com/paper/2311.13608/) (2023)
- [SketchAgent: Language-Driven Sequential Sketch Generation](https://mlsanity.com/paper/2411.17673/) (2024)
- [A Review of the Applications of Deep Learning-Based Emergent Communication](https://mlsanity.com/paper/2407.03302/) (2024)
- [Emergent Graphical Conventions in a Visual Communication Game](https://mlsanity.com/paper/2111.14210/) (2021)
- [Distributed Reinforcement Learning for Robot Teams: A Review](https://mlsanity.com/paper/2204.03516/) (2022)
- [Towards More Human-like AI Communication: A Review of Emergent Communication Research](https://mlsanity.com/paper/2308.02541/) (2023)
- [Multi-Object Sketch Animation with Grouping and Motion Trajectory Priors](https://mlsanity.com/paper/2508.15535/) (2025)
- [Computational Hermeneutics: Evaluating generative AI as a cultural technology](https://mlsanity.com/paper/2604.16403/) (2026)
- [Learning Geometry-aware Representations by Sketching](https://mlsanity.com/paper/2304.08204/) (2023)
- [Emergent Communication in Interactive Sketch Question Answering](https://mlsanity.com/paper/2310.15597/) (2023)
- [ELCC: the Emergent Language Corpus Collection](https://mlsanity.com/paper/2407.04158/) (2024)
- [VideoSketcher: Sequential Sketch Generation Using Video Model Priors](https://mlsanity.com/paper/2602.15819/) (2026)
- [Contrastive Multimodal Learning for Emergence of Graphical Sensory-Motor Communication](https://mlsanity.com/paper/2210.06468/) (2022)
- [XferBench: a Data-Driven Benchmark for Emergent Language](https://mlsanity.com/paper/2407.03456/) (2024)
- [Visual Theory of Mind Enables the Invention of Proto-Writing](https://mlsanity.com/paper/2502.01568/) (2025)
- [Learning to Communicate Across Modalities: Perceptual Heterogeneity in Multi-Agent Systems](https://mlsanity.com/paper/2601.22041/) (2026)
- [Shared Visual Representations of Drawing for Communication: How do different biases affect human interpretability and intent?](https://mlsanity.com/paper/2110.08203/) (2021)
- [Generative Visual Communication in the Era of Vision-Language Models](https://mlsanity.com/paper/2411.18727/) (2024)
- [VQEL: Enabling Self-Play in Emergent Language Games via Agent-Internal Vector Quantization](https://mlsanity.com/paper/2503.04940/) (2025)
- [The Image Reconstruction Game: Drawing Common Ground Through Iterative Multimodal Dialogue](https://mlsanity.com/paper/2606.01901/) (2026)
- [Drawing with Strangers: Population Scaling Drives Zero-Shot Mutual Intelligibility in Emergent Sketching](https://mlsanity.com/paper/2606.10582/) (2026)

## Abstract (arXiv)

Evidence that visual communication preceded written language and provided a basis for it goes back to prehistory, in forms such as cave and rock paintings depicting traces of our distant ancestors. Emergent communication research has sought to explore how agents can learn to communicate in order to collaboratively solve tasks. Existing research has focused on language, with a learned communication channel transmitting sequences of discrete tokens between the agents. In this work, we explore a visual communication channel between agents that are allowed to draw with simple strokes. Our agents are parameterised by deep neural networks, and the drawing procedure is differentiable, allowing for end-to-end training. In the framework of a referential communication game, we demonstrate that agents can not only successfully learn to communicate by drawing, but with appropriate inductive biases, can do so in a fashion that humans can interpret. We hope to encourage future research to consider visual communication as a more flexible and directly interpretable alternative of training collaborative agents.