# A surprisal--duration trade-off across and within the world's languages

arXiv: https://arxiv.org/abs/2109.15000 · HTML: https://mlsanity.com/paper/2109.15000/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 58 references that are in the map.
- First posted on arXiv 30 Sep 2021.

## Builds on (most cited first, 8 of 8)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [On the Predictive Power of Neural Language Models for Human Real-Time Comprehension Behavior](https://mlsanity.com/paper/2006.01912/) (2020)
- [Sharp Nearby, Fuzzy Far Away: How Neural Language Models Use Context](https://mlsanity.com/paper/1805.04623/) (2018)
- [Revisiting the Uniform Information Density Hypothesis](https://mlsanity.com/paper/2109.11635/) (2021)
- [Phonotactic Complexity and its Trade-offs](https://mlsanity.com/paper/2005.03774/) (2020)
- [On the Complexity and Typology of Inflectional Morphological Systems](https://mlsanity.com/paper/1807.02747/) (2018)
- [A Corpus for Large-Scale Phonetic Typology](https://mlsanity.com/paper/2005.13962/) (2020)
- [Disambiguatory Signals are Stronger in Word-initial Positions](https://mlsanity.com/paper/2102.02183/) (2021)

## Built on by (most cited first, 23 of 23)

- [Locally Typical Sampling](https://mlsanity.com/paper/2202.00666/) (2022)
- [Testing the Predictions of Surprisal Theory in 11 Languages](https://mlsanity.com/paper/2307.03667/) (2023)
- [Information Value: Measuring Utterance Predictability as Distance from Plausible Alternatives](https://mlsanity.com/paper/2310.13676/) (2023)
- [A Cross-Linguistic Pressure for Uniform Information Density in Word Order](https://mlsanity.com/paper/2306.03734/) (2023)
- [On the probability-quality paradox in language generation](https://mlsanity.com/paper/2203.17217/) (2022)
- [An Information-Theoretic Analysis of Self-supervised Discrete Representations of Speech](https://mlsanity.com/paper/2306.02405/) (2023)
- [Quantifying the redundancy between prosody and text](https://mlsanity.com/paper/2311.17233/) (2023)
- [The Roots of Performance Disparity in Multilingual Language Models: Intrinsic Modeling Difficulty or Design Choices?](https://mlsanity.com/paper/2601.07220/) (2026)
- [Using Linguistic Typology to Enrich Multilingual Lexicons: the Case of Lexical Gaps in Kinship](https://mlsanity.com/paper/2204.05049/) (2022)
- [Grammatical cues to subjecthood are redundant in a majority of simple clauses across languages](https://mlsanity.com/paper/2201.12911/) (2022)
- [Revisiting the Optimality of Word Lengths](https://mlsanity.com/paper/2312.03897/) (2023)
- [Information-Theoretic Characterization of Vowel Harmony: A Cross-Linguistic Study on Word Lists](https://mlsanity.com/paper/2308.04885/) (2023)
- [The time scale of redundancy between prosody and linguistic context](https://mlsanity.com/paper/2503.11630/) (2025)
- [Computational Sentence-level Metrics Predicting Human Sentence Comprehension](https://mlsanity.com/paper/2403.15822/) (2024)
- [Language models emulate certain cognitive profiles: An investigation of how predictability measures interact with individual differences](https://mlsanity.com/paper/2406.04988/) (2024)
- [Predicting Human Translation Difficulty with Neural Machine Translation](https://mlsanity.com/paper/2312.11852/) (2023)
- [Uniform Information Density and Syntactic Reduction: Revisiting textitthat-Mentioning in English Complement Clauses](https://mlsanity.com/paper/2509.05254/) (2025)
- [HOMURA: Taming the Sand-Glass for Time-Constrained LLM Translation via Reinforcement Learning](https://mlsanity.com/paper/2601.10187/) (2026)
- [The Distribution of Phoneme Frequencies across the World's Languages: Macroscopic and Microscopic Information-Theoretic Models](https://mlsanity.com/paper/2603.02860/) (2026)
- [Surprisal Minimisation over Goal-directed Alternatives Predicts Production Choice in Dialogue](https://mlsanity.com/paper/2605.00506/) (2026)
- [Surprisal reveals diversity gaps in image captioning and different scorers change the story](https://mlsanity.com/paper/2511.04754/) (2025)
- [Modelling the Diachronic Emergence of Phoneme Frequency Distributions](https://mlsanity.com/paper/2603.09503/) (2026)
- [Estimating Surprisal from the Speech Signal](https://mlsanity.com/paper/2610.18439/) (2026)

## Abstract (arXiv)

While there exist scores of natural languages, each with its unique features and idiosyncrasies, they all share a unifying theme: enabling human communication. We may thus reasonably predict that human cognition shapes how these languages evolve and are used. Assuming that the capacity to process information is roughly constant across human populations, we expect a surprisal--duration trade-off to arise both across and within languages. We analyse this trade-off using a corpus of 600 languages and, after controlling for several potential confounds, we find strong supporting evidence in both settings. Specifically, we find that, on average, phones are produced faster in languages where they are less surprising, and vice versa. Further, we confirm that more surprising phones are longer, on average, in 319 languages out of the 600. We thus conclude that there is strong evidence of a surprisal--duration trade-off in operation, both across and within the world's languages.