mlsanityData to 7 Oct 2026

Learning to Draw: Emergent Communication through Sketching

arXiv 2106.02067 ↗First posted 3 Jun 2021cs.CV cs.AI cs.LGcited by 23

Not on the map. Why?

Abstract (arXiv)

Evidence that visual communication preceded written language and provided a basis for it goes back to prehistory, in forms such as cave and rock paintings depicting traces of our distant ancestors. Emergent communication research has sought to explore how agents can learn to communicate in order to collaboratively solve tasks. Existing research has focused on language, with a learned communication channel transmitting sequences of discrete tokens between the agents. In this work, we explore a visual communication channel between agents that are allowed to draw with simple strokes. Our agents are parameterised by deep neural networks, and the drawing procedure is differentiable, allowing for end-to-end training. In the framework of a referential communication game, we demonstrate that agents can not only successfully learn to communicate by drawing, but with appropriate inductive biases, can do so in a fashion that humans can interpret. We hope to encourage future research to consider visual communication as a more flexible and directly interpretable alternative of training collaborative agents.

Key facts. Cited by 23 papers in the map (data through 7 Oct 2026). Builds on 23 of its 50 references that are in the map. First posted on arXiv 3 Jun 2021.

Builds on

23 of 50 references.

References · 23

  1. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  3. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  4. 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
  5. 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,430
  6. 2016Learning to Communicate with Deep Multi-Agent Reinforcement Learningcited by 561
  7. 2017Emergence of Grounded Compositional Language in Multi-Agent Populationscited by 257
  8. 2018TarMAC: Targeted Multi-Agent Communicationcited by 201
  9. 2016Multi-Agent Cooperation and the Emergence of (Natural) Languagecited by 181
  10. 2017Emergence of Language with Multi-agent Games: Learning to Communicate with Sequences of Symbolscited by 123
  11. 2020Compositionality and Generalization in Emergent Languagescited by 103
  12. 2018Emergence of Linguistic Communication from Referential Games with Symbolic and Pixel Inputcited by 102
11 more
  1. 2020Compositional Languages Emerge in a Neural Iterated Learning Modelcited by 75
  2. 2017Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learningcited by 70
  3. 2019On the Pitfalls of Measuring Emergent Communicationcited by 66
  4. 2018Emergent Communication through Negotiationcited by 62
  5. 2019Anti-efficient encoding in emergent communicationcited by 59
  6. 2021Differentiable Drawing and Sketchingcited by 20
  7. 2016Learning to Play Guess Who? and Inventing a Grounded Language as a Consequencecited by 15
  8. 2019Pragmatic inference and visual abstraction enable contextual flexibility during visual communicationcited by 14
  9. 2020Inductive Bias and Language Expressivity in Emergent Communicationcited by 6
  10. 2020From Language Games to Drawing Gamescited by 5
  11. 2019Emergence of Numeric Concepts in Multi-Agent Autonomous Communicationcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 4; 2023: 4; 2024: 5; 2025: 3; 2026: 5.

23 papers.

Sort:

By area:

  1. 2022CLIPascene: Scene Sketching with Different Types and Levels of Abstractioncited by 63
  2. 2022CLIPasso: Semantically-Aware Object Sketchingcited by 57
  3. 2023Breathing Life Into Sketches Using Text-to-Video Priorscited by 47
  4. 2024SketchAgent: Language-Driven Sequential Sketch Generationcited by 30
  5. 2024A Review of the Applications of Deep Learning-Based Emergent Communicationcited by 17
  6. 2021Emergent Graphical Conventions in a Visual Communication Gamecited by 16
  7. 2023Towards More Human-like AI Communication: A Review of Emergent Communication Researchcited by 12
  8. 2022Distributed Reinforcement Learning for Robot Teams: A Reviewcited by 11
  9. 2025Multi-Object Sketch Animation with Grouping and Motion Trajectory Priorscited by 9
  10. 2026Computational Hermeneutics: Evaluating generative AI as a cultural technologycited by 7
  11. 2023Learning Geometry-aware Representations by Sketchingcited by 5
  12. 2023Emergent Communication in Interactive Sketch Question Answeringcited by 3
11 more
  1. 2024ELCC: the Emergent Language Corpus Collectioncited by 2
  2. 2026VideoSketcher: Sequential Sketch Generation Using Video Model Priorscited by 2
  3. 2022Contrastive Multimodal Learning for Emergence of Graphical Sensory-Motor Communicationcited by 1
  4. 2024XferBench: a Data-Driven Benchmark for Emergent Languagecited by 1
  5. 2025Visual Theory of Mind Enables the Invention of Proto-Writingcited by 1
  6. 2026Learning to Communicate Across Modalities: Perceptual Heterogeneity in Multi-Agent Systemscited by 1
  7. 2021Shared Visual Representations of Drawing for Communication: How do different biases affect human interpretability and intent?cited by 0
  8. 2024Generative Visual Communication in the Era of Vision-Language Modelscited by 0
  9. 2025VQEL: Enabling Self-Play in Emergent Language Games via Agent-Internal Vector Quantizationcited by 0
  10. 2026The Image Reconstruction Game: Drawing Common Ground Through Iterative Multimodal Dialoguecited by 0
  11. 2026Drawing with Strangers: Population Scaling Drives Zero-Shot Mutual Intelligibility in Emergent Sketchingcited by 0