# Neural networks can understand compositional functions that humans do not, in the context of emergent communication

arXiv: https://arxiv.org/abs/2103.04180 · HTML: https://mlsanity.com/paper/2103.04180/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 33 references that are in the map.
- First posted on arXiv 6 Mar 2021.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [Learning to Communicate with Deep Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/1605.06676/) (2016)
- [Compositionality decomposed: how do neural networks generalise?](https://mlsanity.com/paper/1908.08351/) (2019)
- [Compositionality and Generalization in Emergent Languages](https://mlsanity.com/paper/2004.09124/) (2020)
- [Emergence of Linguistic Communication from Referential Games with Symbolic and Pixel Input](https://mlsanity.com/paper/1804.03984/) (2018)
- [Natural Language Does Not Emerge 'Naturally' in Multi-Agent Dialog](https://mlsanity.com/paper/1706.08502/) (2017)
- [Measuring Compositionality in Representation Learning](https://mlsanity.com/paper/1902.07181/) (2019)
- [Ease-of-Teaching and Language Structure from Emergent Communication](https://mlsanity.com/paper/1906.02403/) (2019)
- [Simple Recurrent Units for Highly Parallelizable Recurrence](https://mlsanity.com/paper/1709.02755/) (2017)
- [Capacity, Bandwidth, and Compositionality in Emergent Language Learning](https://mlsanity.com/paper/1910.11424/) (2019)
- [Co-evolution of language and agents in referential games](https://mlsanity.com/paper/2001.03361/) (2020)
- [Emergent Language Generalization and Acquisition Speed are not tied to Compositionality](https://mlsanity.com/paper/2004.03420/) (2020)

## Built on by (most cited first, 4 of 4)

- [A Review of the Applications of Deep Learning-Based Emergent Communication](https://mlsanity.com/paper/2407.03302/) (2024)
- [The Five Ws of Multi-Agent Communication: Who Talks to Whom, When, What, and Why -- A Survey from MARL to Emergent Language and LLMs](https://mlsanity.com/paper/2602.11583/) (2026)
- [Speaking Your Language: Spatial Relationships in Interpretable Emergent Communication](https://mlsanity.com/paper/2406.07277/) (2024)
- [ELCC: the Emergent Language Corpus Collection](https://mlsanity.com/paper/2407.04158/) (2024)

## Abstract (arXiv)

We show that it is possible to craft transformations that, applied to compositional grammars, result in grammars that neural networks can learn easily, but humans do not. This could explain the disconnect between current metrics of compositionality, that are arguably human-centric, and the ability of neural networks to generalize to unseen examples. We propose to use the transformations as a benchmark, ICY, which could be used to measure aspects of the compositional inductive bias of networks, and to search for networks with similar compositional inductive biases to humans. As an example of this approach, we propose a hierarchical model, HU-RNN, which shows an inductive bias towards position-independent, word-like groups of tokens.