# Word-order biases in deep-agent emergent communication

arXiv: https://arxiv.org/abs/1905.12330 · HTML: https://mlsanity.com/paper/1905.12330/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 May 2019.

## Built on by (most cited first, 18 of 18)

- [A Review of the Applications of Deep Learning-Based Emergent Communication](https://mlsanity.com/paper/2407.03302/) (2024)
- [Communicating Activations Between Language Model Agents](https://mlsanity.com/paper/2501.14082/) (2025)
- [Towards More Human-like AI Communication: A Review of Emergent Communication Research](https://mlsanity.com/paper/2308.02541/) (2023)
- [Communication Drives the Emergence of Language Universals in Neural Agents: Evidence from the Word-order/Case-marking Trade-off](https://mlsanity.com/paper/2301.13083/) (2023)
- [Emergent Word Order Universals from Cognitively-Motivated Language Models](https://mlsanity.com/paper/2402.12363/) (2024)
- [Learning Multi-Agent Communication with Contrastive Learning](https://mlsanity.com/paper/2307.01403/) (2023)
- [On the Difficulty of Translating Free-Order Case-Marking Languages](https://mlsanity.com/paper/2107.06055/) (2021)
- [NeLLCom-X: A Comprehensive Neural-Agent Framework to Simulate Language Learning and Group Communication](https://mlsanity.com/paper/2407.13999/) (2024)
- [The Curious Case of Representational Alignment: Unravelling Visio-Linguistic Tasks in Emergent Communication](https://mlsanity.com/paper/2407.17960/) (2024)
- [The Effect of Efficient Messaging and Input Variability on Neural-Agent Iterated Language Learning](https://mlsanity.com/paper/2104.07637/) (2021)
- [Visual Referential Games Further the Emergence of Disentangled Representations](https://mlsanity.com/paper/2304.14511/) (2023)
- [Learning to translate by learning to communicate](https://mlsanity.com/paper/2207.07025/) (2022)
- [Meta-Referential Games to Learn Compositional Learning Behaviours](https://mlsanity.com/paper/2207.08012/) (2022)
- [Emergent Communication with Attention](https://mlsanity.com/paper/2305.10920/) (2023)
- [SimSiam Naming Game: A Unified Approach for Emergent Communication and Representation Learning](https://mlsanity.com/paper/2410.21803/) (2024)
- [ETHER: Aligning Emergent Communication for Hindsight Experience Replay](https://mlsanity.com/paper/2307.15494/) (2023)
- [VQEL: Enabling Self-Play in Emergent Language Games via Agent-Internal Vector Quantization](https://mlsanity.com/paper/2503.04940/) (2025)
- [Factors Influencing the Emergence of Dependency Length Minimization in Neural Agent Simulations](https://mlsanity.com/paper/2609.06025/) (2026)

## Abstract (arXiv)

Sequence-processing neural networks led to remarkable progress on many NLP tasks. As a consequence, there has been increasing interest in understanding to what extent they process language as humans do. We aim here to uncover which biases such models display with respect to "natural" word-order constraints. We train models to communicate about paths in a simple gridworld, using miniature languages that reflect or violate various natural language trends, such as the tendency to avoid redundancy or to minimize long-distance dependencies. We study how the controlled characteristics of our miniature languages affect individual learning and their stability across multiple network generations. The results draw a mixed picture. On the one hand, neural networks show a strong tendency to avoid long-distance dependencies. On the other hand, there is no clear preference for the efficient, non-redundant encoding of information that is widely attested in natural language. We thus suggest inoculating a notion of "effort" into neural networks, as a possible way to make their linguistic behavior more human-like.