# Emergent Communication: Generalization and Overfitting in Lewis Games

arXiv: https://arxiv.org/abs/2209.15342 · HTML: https://mlsanity.com/paper/2209.15342/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 90 references that are in the map.
- First posted on arXiv 30 Sep 2022.

## Builds on (most cited first, 30 of 33)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Asynchronous Methods for Deep Reinforcement Learning](https://mlsanity.com/paper/1602.01783/) (2016)
- [Multi-Agent Cooperation and the Emergence of (Natural) Language](https://mlsanity.com/paper/1612.07182/) (2016)
- [Emergent Multi-Agent Communication in the Deep Learning Era](https://mlsanity.com/paper/2006.02419/) (2020)
- [Emergence of Language with Multi-agent Games: Learning to Communicate with Sequences of Symbols](https://mlsanity.com/paper/1705.11192/) (2017)
- [Compositionality and Generalization in Emergent Languages](https://mlsanity.com/paper/2004.09124/) (2020)
- [Emergence of Linguistic Communication from Referential Games with Symbolic and Pixel Input](https://mlsanity.com/paper/1804.03984/) (2018)
- [Natural Language Does Not Emerge 'Naturally' in Multi-Agent Dialog](https://mlsanity.com/paper/1706.08502/) (2017)
- [Measuring Compositionality in Representation Learning](https://mlsanity.com/paper/1902.07181/) (2019)
- [Compositional Languages Emerge in a Neural Iterated Learning Model](https://mlsanity.com/paper/2002.01365/) (2020)
- [Ease-of-Teaching and Language Structure from Emergent Communication](https://mlsanity.com/paper/1906.02403/) (2019)
- [Anti-efficient encoding in emergent communication](https://mlsanity.com/paper/1905.12561/) (2019)
- [Linguistic generalization and compositionality in modern artificial neural networks](https://mlsanity.com/paper/1904.00157/) (2019)
- [EGG: a toolkit for research on Emergence of lanGuage in Games](https://mlsanity.com/paper/1907.00852/) (2019)
- [Emergent Communication of Generalizations](https://mlsanity.com/paper/2106.02668/) (2021)
- [Emergent Linguistic Phenomena in Multi-Agent Communication Games](https://mlsanity.com/paper/1901.08706/) (2019)
- [Capacity, Bandwidth, and Compositionality in Emergent Language Learning](https://mlsanity.com/paper/1910.11424/) (2019)
- [Interpretable agent communication from scratch (with a generic visual processor emerging on the side)](https://mlsanity.com/paper/2106.04258/) (2021)
- [On the role of population heterogeneity in emergent communication](https://mlsanity.com/paper/2204.12982/) (2022)
- ["LazImpa": Lazy and Impatient neural agents learn to communicate efficiently](https://mlsanity.com/paper/2010.01878/) (2020)
- [Emergent Communication for Understanding Human Language Evolution: What's Missing?](https://mlsanity.com/paper/2204.10590/) (2022)
- [CNNs found to jump around more skillfully than RNNs: Compositional generalization in seq2seq convolutional networks](https://mlsanity.com/paper/1905.08527/) (2019)
- [Emergent Communication under Competition](https://mlsanity.com/paper/2101.10276/) (2021)
- [Emergence of Compositional Language with Deep Generational Transmission](https://mlsanity.com/paper/1904.09067/) (2019)
- [Emergent Language Generalization and Acquisition Speed are not tied to Compositionality](https://mlsanity.com/paper/2004.03420/) (2020)

## Built on by (most cited first, 28 of 28)

- [Language Model Alignment with Elastic Reset](https://mlsanity.com/paper/2312.07551/) (2023)
- [A Review of the Applications of Deep Learning-Based Emergent Communication](https://mlsanity.com/paper/2407.03302/) (2024)
- [Generative Emergent Communication: Large Language Model is a Collective World Model](https://mlsanity.com/paper/2501.00226/) (2024)
- [What makes a language easy to deep-learn? Deep neural networks and humans similarly benefit from compositional structure](https://mlsanity.com/paper/2302.12239/) (2023)
- [Communication Drives the Emergence of Language Universals in Neural Agents: Evidence from the Word-order/Case-marking Trade-off](https://mlsanity.com/paper/2301.13083/) (2023)
- [Lewis's Signaling Game as beta-VAE For Natural Word Lengths and Segments](https://mlsanity.com/paper/2311.04453/) (2023)
- [Emergent Word Order Universals from Cognitively-Motivated Language Models](https://mlsanity.com/paper/2402.12363/) (2024)
- [NeLLCom-X: A Comprehensive Neural-Agent Framework to Simulate Language Learning and Group Communication](https://mlsanity.com/paper/2407.13999/) (2024)
- [The Curious Case of Representational Alignment: Unravelling Visio-Linguistic Tasks in Emergent Communication](https://mlsanity.com/paper/2407.17960/) (2024)
- [Basic syntax from speech: Spontaneous concatenation in unsupervised deep neural networks](https://mlsanity.com/paper/2305.01626/) (2023)
- [Language Evolution with Deep Learning](https://mlsanity.com/paper/2403.11958/) (2024)
- [Emergent Communication with Attention](https://mlsanity.com/paper/2305.10920/) (2023)
- [ELCC: the Emergent Language Corpus Collection](https://mlsanity.com/paper/2407.04158/) (2024)
- [Semantics and Spatiality of Emergent Communication](https://mlsanity.com/paper/2411.10173/) (2024)
- [On the Correspondence between Compositionality and Imitation in Emergent Neural Communication](https://mlsanity.com/paper/2305.12941/) (2023)
- [A Compressive-Expressive Communication Framework for Compositional Representations](https://mlsanity.com/paper/2501.19182/) (2025)
- [Wanting to be Understood](https://mlsanity.com/paper/2504.06611/) (2025)
- [Emergent Natural Language with Communication Games for Improving Image Captioning Capabilities without Additional Data](https://mlsanity.com/paper/2507.08610/) (2025)
- [CtD: Composition through Decomposition in Emergent Communication](https://mlsanity.com/paper/2601.10169/) (2026)
- [What You Can't See Is What You Learn: Slot-Selective Evidence Masking Favors Compositional Generalization in Shared-Genome Language-Model Societies](https://mlsanity.com/paper/2608.20054/) (2026)
- [Emergent Communication for Rules Reasoning](https://mlsanity.com/paper/2311.04474/) (2023)
- [Cognitively-Inspired Emergent Communication via Knowledge Graphs for Assisting the Visually Impaired](https://mlsanity.com/paper/2505.22087/) (2025)
- [Why Are Parsing Actions for Understanding Message Hierarchies Not Random?](https://mlsanity.com/paper/2506.22366/) (2025)
- [Successful Misunderstandings: Learning to Coordinate Without Being Understood](https://mlsanity.com/paper/2509.24660/) (2025)
- [Translating Signals to Languages for sEMG-Based Activity Recognition](https://mlsanity.com/paper/2605.22403/) (2026)
- [Drawing with Strangers: Population Scaling Drives Zero-Shot Mutual Intelligibility in Emergent Sketching](https://mlsanity.com/paper/2606.10582/) (2026)
- [Xeno-Interpretability: Investigating the Alien Minds of LLMs](https://mlsanity.com/paper/2609.20408/) (2026)
- [Understanding Confabulation and Rethinking Reconstruction in Activation Explanations](https://mlsanity.com/paper/2609.33702/) (2026)

## Abstract (arXiv)

Lewis signaling games are a class of simple communication games for simulating the emergence of language. In these games, two agents must agree on a communication protocol in order to solve a cooperative task. Previous work has shown that agents trained to play this game with reinforcement learning tend to develop languages that display undesirable properties from a linguistic point of view (lack of generalization, lack of compositionality, etc). In this paper, we aim to provide better understanding of this phenomenon by analytically studying the learning problem in Lewis games. As a core contribution, we demonstrate that the standard objective in Lewis games can be decomposed in two components: a co-adaptation loss and an information loss. This decomposition enables us to surface two potential sources of overfitting, which we show may undermine the emergence of a structured communication protocol. In particular, when we control for overfitting on the co-adaptation loss, we recover desired properties in the emergent languages: they are more compositional and generalize better.