Local word statistics affect reading times independently of surprisal
Not on the map. Why?
Abstract (arXiv)
Surprisal theory has provided a unifying framework for understanding many phenomena in sentence processing (Hale, 2001; Levy, 2008a), positing that a word's conditional probability given all prior context fully determines processing difficulty. Problematically for this claim, one local statistic, word frequency, has also been shown to affect processing, even when conditional probability given context is held constant. Here, we ask whether other local statistics have a role in processing, or whether word frequency is a special case. We present the first clear evidence that more complex local statistics, word bigram and trigram probability, also affect processing independently of surprisal. These findings suggest a significant and independent role of local statistics in processing. Further, it motivates research into new generalizations of surprisal that can also explain why local statistical information should have an outsized effect.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 2 of its 30 references that are in the map. First posted on arXiv 7 Mar 2021.
Builds on
2 of 30 references.
References · 2
- 2013One Billion Word Benchmark for Measuring Progress in Statistical Language Modelingcited by 308
- 2016Exploring the Limits of Language Modelingcited by 173
Built on by
Citing papers per year: 2022: 1; 2023: 0; 2024: 1; 2025: 0; 2026: 1.
3 papers.
- 2022Event knowledge in large language models: the gap between the impossible and the unlikelycited by 45
- 2024Frequency Explains the Inverse Correlation of Large Language Models' Size, Training Data Amount, and Surprisal's Fit to Reading Timescited by 39
- 2026N-gram-like Language Models Predict Naturalistic Reading Time Bestcited by 2
- 2026N-gram-like Language Models Predict Naturalistic Reading Time Best10 Mar 2026
- 2024Frequency Explains the Inverse Correlation of Large Language Models' Size, Training Data Amount, and Surprisal's Fit to Reading Times3 Feb 2024
- 2022Event knowledge in large language models: the gap between the impossible and the unlikely2 Dec 2022