# The cognitive roots of regularization in language

arXiv: https://arxiv.org/abs/1703.03442 · HTML: https://mlsanity.com/paper/1703.03442/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Mar 2017.

## Built on by (most cited first, 5 of 5)

- [Bias Amplification in Language Model Evolution: An Iterated Learning Perspective](https://mlsanity.com/paper/2404.04286/) (2024)
- [From Interpolation to Extrapolation: Complete Length Generalization for Arithmetic Transformers](https://mlsanity.com/paper/2310.11984/) (2023)
- [The Effect of Efficient Messaging and Input Variability on Neural-Agent Iterated Language Learning](https://mlsanity.com/paper/2104.07637/) (2021)
- [Model Collapse as Cultural Evolution](https://mlsanity.com/paper/2605.23054/) (2026)
- [AI Pedagogy: Dialogic Social Learning for Artificial Agents](https://mlsanity.com/paper/2507.21065/) (2025)

## Abstract (arXiv)

Regularization occurs when the output a learner produces is less variable than the linguistic data they observed. In an artificial language learning experiment, we show that there exist at least two independent sources of regularization bias in cognition: a domain-general source based on cognitive load and a domain-specific source triggered by linguistic stimuli. Both of these factors modulate how frequency information is encoded and produced, but only the production-side modulations result in regularization (i.e. cause learners to eliminate variation from the observed input). We formalize the definition of regularization as the reduction of entropy and find that entropy measures are better at identifying regularization behavior than frequency-based analyses. Using our experimental data and a model of cultural transmission, we generate predictions for the amount of regularity that would develop in each experimental condition if the artificial language were transmitted over several generations of learners. Here we find that the effect of cognitive constraints can become more complex when put into the context of cultural evolution: although learning biases certainly carry information about the course of language evolution, we should not expect a one-to-one correspondence between the micro-level processes that regularize linguistic datasets and the macro-level evolution of linguistic regularity.