# Compositional Obverter Communication Learning From Raw Visual Input

arXiv: https://arxiv.org/abs/1804.02341 · HTML: https://mlsanity.com/paper/1804.02341/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Apr 2018.

## Built on by (most cited first, 30 of 30)

- [Emergent Communication of Generalizations](https://mlsanity.com/paper/2106.02668/) (2021)
- [Learning to Ground Multi-Agent Communication with Autoencoders](https://mlsanity.com/paper/2110.15349/) (2021)
- [Interpretable agent communication from scratch (with a generic visual processor emerging on the side)](https://mlsanity.com/paper/2106.04258/) (2021)
- [Towards More Human-like AI Communication: A Review of Emergent Communication Research](https://mlsanity.com/paper/2308.02541/) (2023)
- [Catalytic Role Of Noise And Necessity Of Inductive Biases In The Emergence Of Compositional Communication](https://mlsanity.com/paper/2111.06464/) (2021)
- [Lewis's Signaling Game as beta-VAE For Natural Word Lengths and Segments](https://mlsanity.com/paper/2311.04453/) (2023)
- [The Five Ws of Multi-Agent Communication: Who Talks to Whom, When, What, and Why -- A Survey from MARL to Emergent Language and LLMs](https://mlsanity.com/paper/2602.11583/) (2026)
- [Information Design in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2305.06807/) (2023)
- [Emergence of Theory of Mind Collaboration in Multiagent Systems](https://mlsanity.com/paper/2110.00121/) (2021)
- [Emergence of hierarchical reference systems in multi-agent communication](https://mlsanity.com/paper/2203.13176/) (2022)
- [Recursive Metropolis-Hastings Naming Game: Symbol Emergence in a Multi-agent System based on Probabilistic Generative Models](https://mlsanity.com/paper/2305.19761/) (2023)
- [Emergent Quantized Communication](https://mlsanity.com/paper/2211.02412/) (2022)
- [Visual Referential Games Further the Emergence of Disentangled Representations](https://mlsanity.com/paper/2304.14511/) (2023)
- [Learning to Communicate with Strangers via Channel Randomisation Methods](https://mlsanity.com/paper/2104.09557/) (2021)
- [Over-communicate no more: Situated RL agents learn concise communication protocols](https://mlsanity.com/paper/2211.01480/) (2022)
- [Learning Multi-Object Positional Relationships via Emergent Communication](https://mlsanity.com/paper/2302.08084/) (2023)
- [An iterated learning model of language change that mixes supervised and unsupervised learning](https://mlsanity.com/paper/2405.20818/) (2024)
- [Unsupervised Translation of Emergent Communication](https://mlsanity.com/paper/2502.07552/) (2025)
- [Meta-Referential Games to Learn Compositional Learning Behaviours](https://mlsanity.com/paper/2207.08012/) (2022)
- [Models of symbol emergence in communication: a conceptual review and a guide for avoiding local minima](https://mlsanity.com/paper/2303.04544/) (2023)
- [Semantics and Spatiality of Emergent Communication](https://mlsanity.com/paper/2411.10173/) (2024)
- [Implicit Repair with Reinforcement Learning in Emergent Communication](https://mlsanity.com/paper/2502.12624/) (2025)
- [Investigating the Development of Task-Oriented Communication in Vision-Language Models](https://mlsanity.com/paper/2601.20641/) (2026)
- [ClusterComm: Discrete Communication in Decentralized MARL using Internal Representation Clustering](https://mlsanity.com/paper/2401.03504/) (2024)
- [CtD: Composition through Decomposition in Emergent Communication](https://mlsanity.com/paper/2601.10169/) (2026)
- [Emergent Compositional Communication for Latent World Properties](https://mlsanity.com/paper/2604.03266/) (2026)
- [Describe me an Aucklet: Generating Grounded Perceptual Category Descriptions](https://mlsanity.com/paper/2303.04053/) (2023)
- [ETHER: Aligning Emergent Communication for Hindsight Experience Replay](https://mlsanity.com/paper/2307.15494/) (2023)
- [Emergent Communication in Continuous Worlds: Self-Organisation of Conceptually Grounded Vocabularies at Scale](https://mlsanity.com/paper/2401.08461/) (2024)
- [Drawing with Strangers: Population Scaling Drives Zero-Shot Mutual Intelligibility in Emergent Sketching](https://mlsanity.com/paper/2606.10582/) (2026)

## Abstract (arXiv)

One of the distinguishing aspects of human language is its compositionality, which allows us to describe complex environments with limited vocabulary. Previously, it has been shown that neural network agents can learn to communicate in a highly structured, possibly compositional language based on disentangled input (e.g. hand- engineered features). Humans, however, do not learn to communicate based on well-summarized features. In this work, we train neural agents to simultaneously develop visual perception from raw image pixels, and learn to communicate with a sequence of discrete symbols. The agents play an image description game where the image contains factors such as colors and shapes. We train the agents using the obverter technique where an agent introspects to generate messages that maximize its own understanding. Through qualitative analysis, visualization and a zero-shot test, we show that the agents can develop, out of raw image pixels, a language with compositional properties, given a proper pressure from the environment.