# Language Models Explain Word Reading Times Better Than Empirical Predictability

arXiv: https://arxiv.org/abs/2202.01128 · HTML: https://mlsanity.com/paper/2202.01128/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 87 references that are in the map.
- First posted on arXiv 2 Feb 2022.

## Builds on (most cited first, 4 of 4)

- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Advances in Pre-Training Distributed Word Representations](https://mlsanity.com/paper/1712.09405/) (2017)
- [On the Predictive Power of Neural Language Models for Human Real-Time Comprehension Behavior](https://mlsanity.com/paper/2006.01912/) (2020)
- [Individual corpora predict fast memory retrieval during reading](https://mlsanity.com/paper/2010.10176/) (2020)

## Built on by (most cited first, 9 of 9)

- [Probing Large Language Models from A Human Behavioral Perspective](https://mlsanity.com/paper/2310.05216/) (2023)
- [Large Language Models for Psycholinguistic Plausibility Pretesting](https://mlsanity.com/paper/2402.05455/) (2024)
- [Shades of Zero: Distinguishing Impossibility from Inconceivability](https://mlsanity.com/paper/2502.20469/) (2025)
- [Individual Text Corpora Predict Openness, Interests, Knowledge and Level of Education](https://mlsanity.com/paper/2404.00165/) (2024)
- [Clozing the Gap: Exploring Why Language Model Surprisal Outperforms Cloze Surprisal](https://mlsanity.com/paper/2601.09886/) (2026)
- [LLM surprisal is necessary but not sufficient to capture English garden-path effects: Evidence from joint latent modeling of reading paradigms](https://mlsanity.com/paper/2602.04489/) (2026)
- [Predictability and Causality in Spanish and English Natural Language Generation](https://mlsanity.com/paper/2408.14283/) (2024)
- [Modelando procesos cognitivos de la lectura natural con GPT-2](https://mlsanity.com/paper/2409.20174/) (2024)
- [Modeling cognitive processes of natural reading with transformer-based Language Models](https://mlsanity.com/paper/2505.11485/) (2025)

## Abstract (arXiv)

Though there is a strong consensus that word length and frequency are the most important single-word features determining visual-orthographic access to the mental lexicon, there is less agreement as how to best capture syntactic and semantic factors. The traditional approach in cognitive reading research assumes that word predictability from sentence context is best captured by cloze completion probability (CCP) derived from human performance data. We review recent research suggesting that probabilistic language models provide deeper explanations for syntactic and semantic effects than CCP. Then we compare CCP with (1) Symbolic n-gram models consolidate syntactic and semantic short-range relations by computing the probability of a word to occur, given two preceding words. (2) Topic models rely on subsymbolic representations to capture long-range semantic similarity by word co-occurrence counts in documents. (3) In recurrent neural networks (RNNs), the subsymbolic units are trained to predict the next word, given all preceding words in the sentences. To examine lexical retrieval, these models were used to predict single fixation durations and gaze durations to capture rapidly successful and standard lexical access, and total viewing time to capture late semantic integration. The linear item-level analyses showed greater correlations of all language models with all eye-movement measures than CCP. Then we examined non-linear relations between the different types of predictability and the reading times using generalized additive models. N-gram and RNN probabilities of the present word more consistently predicted reading performance compared with topic models or CCP.