The cognitive roots of regularization in language
Not on the map. Why?
Abstract (arXiv)
Regularization occurs when the output a learner produces is less variable than the linguistic data they observed. In an artificial language learning experiment, we show that there exist at least two independent sources of regularization bias in cognition: a domain-general source based on cognitive load and a domain-specific source triggered by linguistic stimuli. Both of these factors modulate how frequency information is encoded and produced, but only the production-side modulations result in regularization (i.e. cause learners to eliminate variation from the observed input). We formalize the definition of regularization as the reduction of entropy and find that entropy measures are better at identifying regularization behavior than frequency-based analyses. Using our experimental data and a model of cultural transmission, we generate predictions for the amount of regularity that would develop in each experimental condition if the artificial language were transmitted over several generations of learners. Here we find that the effect of cognitive constraints can become more complex when put into the context of cultural evolution: although learning biases certainly carry information about the course of language evolution, we should not expect a one-to-one correspondence between the micro-level processes that regularize linguistic datasets and the macro-level evolution of linguistic regularity.
Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). First posted on arXiv 9 Mar 2017.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 1; 2022: 0; 2023: 1; 2024: 1; 2025: 1; 2026: 1.
5 papers.
- 2024Bias Amplification in Language Model Evolution: An Iterated Learning Perspectivecited by 17
- 2023From Interpolation to Extrapolation: Complete Length Generalization for Arithmetic Transformerscited by 10
- 2021The Effect of Efficient Messaging and Input Variability on Neural-Agent Iterated Language Learningcited by 6
- 2026Model Collapse as Cultural Evolutioncited by 1
- 2025AI Pedagogy: Dialogic Social Learning for Artificial Agentscited by 0
- 2026Model Collapse as Cultural Evolution21 May 2026
- 2025AI Pedagogy: Dialogic Social Learning for Artificial Agents25 May 2025
- 2024Bias Amplification in Language Model Evolution: An Iterated Learning Perspective4 Apr 2024
- 2023From Interpolation to Extrapolation: Complete Length Generalization for Arithmetic Transformers18 Oct 2023
- 2021The Effect of Efficient Messaging and Input Variability on Neural-Agent Iterated Language Learning15 Apr 2021