# On the Effect of Anticipation on Reading Times

arXiv: https://arxiv.org/abs/2211.14301 · HTML: https://mlsanity.com/paper/2211.14301/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 62 references that are in the map.
- First posted on arXiv 25 Nov 2022.

## Builds on (most cited first, 9 of 9)

- [On the Predictive Power of Neural Language Models for Human Real-Time Comprehension Behavior](https://mlsanity.com/paper/2006.01912/) (2020)
- [Revisiting the Uniform Information Density Hypothesis](https://mlsanity.com/paper/2109.11635/) (2021)
- [Lower Perplexity is Not Always Human-Like](https://mlsanity.com/paper/2106.01229/) (2021)
- [The Natural Stories Corpus](https://mlsanity.com/paper/1708.05763/) (2017)
- [Context Limitations Make Neural Language Models More Human-Like](https://mlsanity.com/paper/2205.11463/) (2022)
- [A Neural Model of Adaptation in Reading](https://mlsanity.com/paper/1808.09930/) (2018)
- [Analyzing Wrap-Up Effects through an Information-Theoretic Lens](https://mlsanity.com/paper/2203.17213/) (2022)
- [Can Entropy Explain Successor Surprisal Effects in Reading?](https://mlsanity.com/paper/1810.11481/) (2018)
- [A Deep Learning Approach to Analyzing Continuous-Time Systems](https://mlsanity.com/paper/2209.12128/) (2022)

## Built on by (most cited first, 19 of 19)

- [Testing the Predictions of Surprisal Theory in 11 Languages](https://mlsanity.com/paper/2307.03667/) (2023)
- [How to Compute the Probability of a Word](https://mlsanity.com/paper/2406.14561/) (2024)
- [Large Language Models Are Human-Like Internally](https://mlsanity.com/paper/2502.01615/) (2025)
- [Generalized Measures of Anticipation and Responsivity in Online Language Processing](https://mlsanity.com/paper/2409.10728/) (2024)
- [Temperature-scaling surprisal estimates improve fit to human reading times -- but does it do so for the "right reasons"?](https://mlsanity.com/paper/2311.09325/) (2023)
- [EMTeC: A Corpus of Eye Movements on Machine-Generated Texts](https://mlsanity.com/paper/2408.04289/) (2024)
- [Psychometric Predictive Power of Large Language Models](https://mlsanity.com/paper/2311.07484/) (2023)
- [How Well Does First-Token Entropy Approximate Word Entropy as a Psycholinguistic Predictor?](https://mlsanity.com/paper/2507.22209/) (2025)
- [Language models emulate certain cognitive profiles: An investigation of how predictability measures interact with individual differences](https://mlsanity.com/paper/2406.04988/) (2024)
- [Reverse-Engineering the Reader](https://mlsanity.com/paper/2410.13086/) (2024)
- [Eye Tracking Based Cognitive Evaluation of Automatic Readability Assessment Methods](https://mlsanity.com/paper/2502.11150/) (2025)
- [From Input Perception to Predictive Insight: Modeling Model Blind Spots Before They Become Errors](https://mlsanity.com/paper/2509.20065/) (2025)
- [An Existence Proof for Neural Language Models That Can Explain Garden-Path Effects via Surprisal](https://mlsanity.com/paper/2604.18293/) (2026)
- [That's Optional: A Contemporary Exploration of "that" Omission in English Subordinate Clauses](https://mlsanity.com/paper/2405.20833/) (2024)
- [Modeling Bottom-up Information Quality during Language Processing](https://mlsanity.com/paper/2509.17047/) (2025)
- [Glitter: Visualizing Lexical Surprisal for Readability in Administrative Texts](https://mlsanity.com/paper/2601.05411/) (2026)
- [Modeling semantic association in self-paced reading with language model embeddings](https://mlsanity.com/paper/2606.07066/) (2026)
- [Do Cantonese-Adapted Language Models Better Predict Cantonese Reading? A Cross-Model Eye-Tracking Evaluation](https://mlsanity.com/paper/2609.02163/) (2026)
- [A Systematic Analysis of the Predictive Power of LM Surprisal in Reading Chinese](https://mlsanity.com/paper/2610.04898/) (2026)

## Abstract (arXiv)

Over the past two decades, numerous studies have demonstrated how less predictable (i.e., higher surprisal) words take more time to read. In general, these studies have implicitly assumed the reading process is purely responsive: Readers observe a new word and allocate time to process it as required. We argue that prior results are also compatible with a reading process that is at least partially anticipatory: Readers could make predictions about a future word and allocate time to process it based on their expectation. In this work, we operationalize this anticipation as a word's contextual entropy. We assess the effect of anticipation on reading by comparing how well surprisal and contextual entropy predict reading times on four naturalistic reading datasets: two self-paced and two eye-tracking. Experimentally, across datasets and analyses, we find substantial evidence for effects of contextual entropy over surprisal on a word's reading time (RT): in fact, entropy is sometimes better than surprisal in predicting a word's RT. Spillover effects, however, are generally not captured by entropy, but only by surprisal. Further, we hypothesize four cognitive mechanisms through which contextual entropy could impact RTs -- three of which we are able to design experiments to analyze. Overall, our results support a view of reading that is not just responsive, but also anticipatory.