# Catalytic Role Of Noise And Necessity Of Inductive Biases In The Emergence Of Compositional Communication

arXiv: https://arxiv.org/abs/2111.06464 · HTML: https://mlsanity.com/paper/2111.06464/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 70 references that are in the map.
- First posted on arXiv 11 Nov 2021.

## Builds on (most cited first, 27 of 27)

- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Accurate, Large Minibatch SGD: Training ImageNet in 1 Hour](https://mlsanity.com/paper/1706.02677/) (2017)
- [Challenging Common Assumptions in the Unsupervised Learning of Disentangled Representations](https://mlsanity.com/paper/1811.12359/) (2018)
- [Linear Mode Connectivity and the Lottery Ticket Hypothesis](https://mlsanity.com/paper/1912.05671/) (2019)
- [Disentangling by Factorising](https://mlsanity.com/paper/1802.05983/) (2018)
- [Learning to Communicate with Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1605.06676/) (2016)
- [Emergence of Grounded Compositional Language in Multi-Agent Populations](https://mlsanity.com/paper/1703.04908/) (2017)
- [Multi-Agent Cooperation and the Emergence of (Natural) Language](https://mlsanity.com/paper/1612.07182/) (2016)
- [Language as an Abstraction for Hierarchical Deep Reinforcement Learning](https://mlsanity.com/paper/1906.07343/) (2019)
- [Compositionality and Generalization in Emergent Languages](https://mlsanity.com/paper/2004.09124/) (2020)
- [Emergence of Linguistic Communication from Referential Games with Symbolic and Pixel Input](https://mlsanity.com/paper/1804.03984/) (2018)
- [Natural Language Does Not Emerge 'Naturally' in Multi-Agent Dialog](https://mlsanity.com/paper/1706.08502/) (2017)
- [Compositional Languages Emerge in a Neural Iterated Learning Model](https://mlsanity.com/paper/2002.01365/) (2020)
- [Learning Cooperative Visual Dialog Agents with Deep Reinforcement Learning](https://mlsanity.com/paper/1703.06585/) (2017)
- [Ease-of-Teaching and Language Structure from Emergent Communication](https://mlsanity.com/paper/1906.02403/) (2019)
- [How agents see things: On visual representations in an emergent language game](https://mlsanity.com/paper/1808.10696/) (2018)
- [Compositional Obverter Communication Learning From Raw Visual Input](https://mlsanity.com/paper/1804.02341/) (2018)
- [Capacity, Bandwidth, and Compositionality in Emergent Language Learning](https://mlsanity.com/paper/1910.11424/) (2019)
- [Discrete Autoencoders for Sequence Models](https://mlsanity.com/paper/1801.09797/) (2018)
- [Emergent Translation in Multi-Agent Communication](https://mlsanity.com/paper/1710.06922/) (2017)
- [Emergent Discrete Communication in Semantic Spaces](https://mlsanity.com/paper/2108.01828/) (2021)
- [Emergence of Communication in an Interactive World with Consistent Speakers](https://mlsanity.com/paper/1809.00549/) (2018)
- [Emergence of Compositional Language with Deep Generational Transmission](https://mlsanity.com/paper/1904.09067/) (2019)
- [Measuring non-trivial compositionality in emergent communication](https://mlsanity.com/paper/2010.15058/) (2020)
- [Developmentally motivated emergence of compositional communication via template transfer](https://mlsanity.com/paper/1910.06079/) (2019)
- [Inductive Bias and Language Expressivity in Emergent Communication](https://mlsanity.com/paper/2012.02875/) (2020)
- [Towards Graph Representation Learning in Emergent Communication](https://mlsanity.com/paper/2001.09063/) (2020)

## Built on by (most cited first, 12 of 12)

- [Emergent Communication: Generalization and Overfitting in Lewis Games](https://mlsanity.com/paper/2209.15342/) (2022)
- [A Review of the Applications of Deep Learning-Based Emergent Communication](https://mlsanity.com/paper/2407.03302/) (2024)
- [Towards More Human-like AI Communication: A Review of Emergent Communication Research](https://mlsanity.com/paper/2308.02541/) (2023)
- [Towards Human-Agent Communication via the Information Bottleneck Principle](https://mlsanity.com/paper/2207.00088/) (2022)
- [Linguistic Structure from a Bottleneck on Sequential Information Processing](https://mlsanity.com/paper/2405.12109/) (2024)
- [Emergent Communication in Interactive Sketch Question Answering](https://mlsanity.com/paper/2310.15597/) (2023)
- [Models of symbol emergence in communication: a conceptual review and a guide for avoiding local minima](https://mlsanity.com/paper/2303.04544/) (2023)
- [Emergent Communication with Attention](https://mlsanity.com/paper/2305.10920/) (2023)
- [ELCC: the Emergent Language Corpus Collection](https://mlsanity.com/paper/2407.04158/) (2024)
- [Emergent communication for AR](https://mlsanity.com/paper/2308.07342/) (2023)
- [Cognitively-Inspired Emergent Communication via Knowledge Graphs for Assisting the Visually Impaired](https://mlsanity.com/paper/2505.22087/) (2025)
- [Factors Influencing the Emergence of Dependency Length Minimization in Neural Agent Simulations](https://mlsanity.com/paper/2609.06025/) (2026)

## Abstract (arXiv)

Communication is compositional if complex signals can be represented as a combination of simpler subparts. In this paper, we theoretically show that inductive biases on both the training framework and the data are needed to develop a compositional communication. Moreover, we prove that compositionality spontaneously arises in the signaling games, where agents communicate over a noisy channel. We experimentally confirm that a range of noise levels, which depends on the model and the data, indeed promotes compositionality. Finally, we provide a comprehensive study of this dependence and report results in terms of recently studied compositionality metrics: topographical similarity, conflict count, and context independence.