# Searching for Structure: Investigating Emergent Communication with Large Language Models

arXiv: https://arxiv.org/abs/2412.07646 · HTML: https://mlsanity.com/paper/2412.07646/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 66 references that are in the map.
- First posted on arXiv 10 Dec 2024.

## Builds on (most cited first, 20 of 20)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Generative Agents: Interactive Simulacra of Human Behavior](https://mlsanity.com/paper/2304.03442/) (2023)
- [Lost in the Middle: How Language Models Use Long Contexts](https://mlsanity.com/paper/2307.03172/) (2023)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies](https://mlsanity.com/paper/2208.10264/) (2022)
- [Reframing Instructional Prompts to GPTk's Language](https://mlsanity.com/paper/2109.07830/) (2021)
- [Emergent Multi-Agent Communication in the Deep Learning Era](https://mlsanity.com/paper/2006.02419/) (2020)
- [Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMs](https://mlsanity.com/paper/2309.07311/) (2023)
- [A Note on Shumailov et al. (2024): `AI Models Collapse When Trained on Recursively Generated Data'](https://mlsanity.com/paper/2410.12954/) (2024)
- [What Artificial Neural Networks Can Tell Us About Human Language Acquisition](https://mlsanity.com/paper/2208.07998/) (2022)
- [Auxiliary task demands mask the capabilities of smaller language models](https://mlsanity.com/paper/2404.02418/) (2024)
- [Mind the instructions: a holistic evaluation of consistency and interactions in prompt-based learning](https://mlsanity.com/paper/2310.13486/) (2023)
- [Emergent Communication for Understanding Human Language Evolution: What's Missing?](https://mlsanity.com/paper/2204.10590/) (2022)
- [A Study on Large Language Models' Limitations in Multiple-Choice Question Answering](https://mlsanity.com/paper/2401.07955/) (2024)
- [Large Language Models: The Need for Nuance in Current Debates and a Pragmatic Perspective on Understanding](https://mlsanity.com/paper/2310.19671/) (2023)
- [Communication Drives the Emergence of Language Universals in Neural Agents: Evidence from the Word-order/Case-marking Trade-off](https://mlsanity.com/paper/2301.13083/) (2023)
- [Learning and communication pressures in neural networks: Lessons from emergent communication](https://mlsanity.com/paper/2403.14427/) (2024)
- [NeLLCom-X: A Comprehensive Neural-Agent Framework to Simulate Language Learning and Group Communication](https://mlsanity.com/paper/2407.13999/) (2024)
- [The Curious Case of Representational Alignment: Unravelling Visio-Linguistic Tasks in Emergent Communication](https://mlsanity.com/paper/2407.17960/) (2024)

## Built on by (most cited first, 10 of 10)

- [Agentic Large Language Models, a survey](https://mlsanity.com/paper/2503.23037/) (2025)
- [Teacher Demonstrations in a BabyLM's Zone of Proximal Development for Contingent Multi-Turn Interaction](https://mlsanity.com/paper/2510.20411/) (2025)
- [From Signals to Structure: How Memory Architecture Drives Language Emergence in LLM Agents](https://mlsanity.com/paper/2607.00233/) (2026)
- [Evolution and compression in LLMs: On the emergence of human-aligned categorization](https://mlsanity.com/paper/2509.08093/) (2025)
- [GlossoGen: Emergent Language in Complex Multi-Agent LLM Interactions](https://mlsanity.com/paper/2609.01491/) (2026)
- [When LLMs Develop Languages: Symbolic Communication for Efficient Multi-Agent Reasoning](https://mlsanity.com/paper/2606.29354/) (2026)
- [Lexical discovery in unknown environments orchestrated by Large Language Models](https://mlsanity.com/paper/2607.22591/) (2026)
- [ConlangBench: Exploring Language Knowledge and Learning in LLMs through Diverse Constructed Languages](https://mlsanity.com/paper/2608.03505/) (2026)
- [Factors Influencing the Emergence of Dependency Length Minimization in Neural Agent Simulations](https://mlsanity.com/paper/2609.06025/) (2026)
- [Communication between Frozen Large Language Models via Prompt Optimization in a Referential Game](https://mlsanity.com/paper/2609.31989/) (2026)

## Abstract (arXiv)

Human languages have evolved to be structured through repeated language learning and use. These processes introduce biases that operate during language acquisition and shape linguistic systems toward communicative efficiency. In this paper, we investigate whether the same happens if artificial languages are optimised for implicit biases of Large Language Models (LLMs). To this end, we simulate a classical referential game in which LLMs learn and use artificial languages. Our results show that initially unstructured holistic languages are indeed shaped to have some structural properties that allow two LLM agents to communicate successfully. Similar to observations in human experiments, generational transmission increases the learnability of languages, but can at the same time result in non-humanlike degenerate vocabularies. Taken together, this work extends experimental findings, shows that LLMs can be used as tools in simulations of language evolution, and opens possibilities for future human-machine experiments in this field.