# Capacity, Bandwidth, and Compositionality in Emergent Language Learning

arXiv: https://arxiv.org/abs/1910.11424 · HTML: https://mlsanity.com/paper/1910.11424/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Oct 2019.

## Built on by (most cited first, 30 of 30)

- [A Survey of Multi-Agent Deep Reinforcement Learning with Communication](https://mlsanity.com/paper/2203.08975/) (2022)
- [What Artificial Neural Networks Can Tell Us About Human Language Acquisition](https://mlsanity.com/paper/2208.07998/) (2022)
- [Emergent Communication of Generalizations](https://mlsanity.com/paper/2106.02668/) (2021)
- [Emergent Communication: Generalization and Overfitting in Lewis Games](https://mlsanity.com/paper/2209.15342/) (2022)
- [Emergent Communication for Understanding Human Language Evolution: What's Missing?](https://mlsanity.com/paper/2204.10590/) (2022)
- [Integrating LLMs with ITS: Recent Advances, Potentials, Challenges, and Future Directions](https://mlsanity.com/paper/2501.04437/) (2025)
- [Generative learning for nonlinear dynamics](https://mlsanity.com/paper/2311.04128/) (2023)
- [A Review of the Applications of Deep Learning-Based Emergent Communication](https://mlsanity.com/paper/2407.03302/) (2024)
- [What makes a language easy to deep-learn? Deep neural networks and humans similarly benefit from compositional structure](https://mlsanity.com/paper/2302.12239/) (2023)
- [Towards More Human-like AI Communication: A Review of Emergent Communication Research](https://mlsanity.com/paper/2308.02541/) (2023)
- [Catalytic Role Of Noise And Necessity Of Inductive Biases In The Emergence Of Compositional Communication](https://mlsanity.com/paper/2111.06464/) (2021)
- [Lewis's Signaling Game as beta-VAE For Natural Word Lengths and Segments](https://mlsanity.com/paper/2311.04453/) (2023)
- [The Five Ws of Multi-Agent Communication: Who Talks to Whom, When, What, and Why -- A Survey from MARL to Emergent Language and LLMs](https://mlsanity.com/paper/2602.11583/) (2026)
- [Compositionality as Lexical Symmetry](https://mlsanity.com/paper/2201.12926/) (2022)
- [Emergent Quantized Communication](https://mlsanity.com/paper/2211.02412/) (2022)
- [Multi-Agent Curricula and Emergent Implicit Signaling](https://mlsanity.com/paper/2106.11156/) (2021)
- [Language Evolution with Deep Learning](https://mlsanity.com/paper/2403.11958/) (2024)
- [Neural networks can understand compositional functions that humans do not, in the context of emergent communication](https://mlsanity.com/paper/2103.04180/) (2021)
- [From Signals to Structure: How Memory Architecture Drives Language Emergence in LLM Agents](https://mlsanity.com/paper/2607.00233/) (2026)
- [Towards Learning to Speak and Hear Through Multi-Agent Communication over a Continuous Acoustic Channel](https://mlsanity.com/paper/2111.02827/) (2021)
- [Fully Independent Communication in Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2401.15059/) (2024)
- [Recommendations for Systematic Research on Emergent Language](https://mlsanity.com/paper/2206.11302/) (2022)
- [Mathematically Modeling the Lexicon Entropy of Emergent Language](https://mlsanity.com/paper/2211.15783/) (2022)
- [Modeling Emergent Lexicon Formation with a Self-Reinforcing Stochastic Process](https://mlsanity.com/paper/2206.11146/) (2022)
- [Discrete Messages Improve Communication Efficiency among Isolated Intelligent Agents](https://mlsanity.com/paper/2312.15985/) (2023)
- [What You Can't See Is What You Learn: Slot-Selective Evidence Masking Favors Compositional Generalization in Shared-Genome Language-Model Societies](https://mlsanity.com/paper/2608.20054/) (2026)
- [Discovering Properties of Inflectional Morphology in Neural Emergent Communication](https://mlsanity.com/paper/2508.05843/) (2025)
- [Emergent Language as an Approach to Conscious AI](https://mlsanity.com/paper/2606.06380/) (2026)
- [EmCom-Diffusion: Probing Visual Reflection in Emergent Languages via Image Generation](https://mlsanity.com/paper/2607.03752/) (2026)
- [Robust and Efficient Communication for Multi-Agent Learning](https://mlsanity.com/paper/2609.15361/) (2026)

## Abstract (arXiv)

Many recent works have discussed the propensity, or lack thereof, for emergent languages to exhibit properties of natural languages. A favorite in the literature is learning compositionality. We note that most of those works have focused on communicative bandwidth as being of primary importance. While important, it is not the only contributing factor. In this paper, we investigate the learning biases that affect the efficacy and compositionality of emergent languages. Our foremost contribution is to explore how capacity of a neural network impacts its ability to learn a compositional language. We additionally introduce a set of evaluation metrics with which we analyze the learned languages. Our hypothesis is that there should be a specific range of model capacity and channel bandwidth that induces compositional structure in the resulting language and consequently encourages systematic generalization. While we empirically see evidence for the bottom of this range, we curiously do not find evidence for the top part of the range and believe that this is an open question for the community.