# Probing for Reading Times

arXiv: https://arxiv.org/abs/2604.18712 · HTML: https://mlsanity.com/paper/2604.18712/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 47 references that are in the map.
- First posted on arXiv 20 Apr 2026.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [Eliciting Latent Predictions from Transformers with the Tuned Lens](https://mlsanity.com/paper/2303.08112/) (2023)
- [Do Llamas Work in English? On the Latent Language of Multilingual Transformers](https://mlsanity.com/paper/2402.10588/) (2024)
- [On the Predictive Power of Neural Language Models for Human Real-Time Comprehension Behavior](https://mlsanity.com/paper/2006.01912/) (2020)
- [Why Does Surprisal From Larger Transformer-Based Language Models Provide a Poorer Fit to Human Reading Times?](https://mlsanity.com/paper/2212.12131/) (2022)
- [Testing the Predictions of Surprisal Theory in 11 Languages](https://mlsanity.com/paper/2307.03667/) (2023)
- [Why do language models perform worse for morphologically complex languages?](https://mlsanity.com/paper/2411.14198/) (2024)
- [How to Compute the Probability of a Word](https://mlsanity.com/paper/2406.14561/) (2024)
- [Everything, Everywhere, All at Once: Is Mechanistic Interpretability Identifiable?](https://mlsanity.com/paper/2502.20914/) (2025)
- [A Non-Linear Structural Probe](https://mlsanity.com/paper/2105.10185/) (2021)
- [From Language Models over Tokens to Language Models over Characters](https://mlsanity.com/paper/2412.03719/) (2024)
- [Leading Whitespaces of Language Models' Subword Vocabulary Pose a Confound for Calculating Word Probabilities](https://mlsanity.com/paper/2406.10851/) (2024)
- [Large Language Models Are Human-Like Internally](https://mlsanity.com/paper/2502.01615/) (2025)
- [Generalized Measures of Anticipation and Responsivity in Online Language Processing](https://mlsanity.com/paper/2409.10728/) (2024)
- [Information Value: Measuring Utterance Predictability as Distance from Plausible Alternatives](https://mlsanity.com/paper/2310.13676/) (2023)
- [The Foundations of Tokenization: Statistical and Computational Concerns](https://mlsanity.com/paper/2407.11606/) (2024)
- [On the Role of Context in Reading Time Prediction](https://mlsanity.com/paper/2409.08160/) (2024)
- [Analyzing Wrap-Up Effects through an Information-Theoretic Lens](https://mlsanity.com/paper/2203.17213/) (2022)
- [Introducing cosmosGPT: Monolingual Training for Turkish Language Models](https://mlsanity.com/paper/2404.17336/) (2024)
- [A Spatio-Temporal Point Process for Fine-Grained Modeling of Reading Behavior](https://mlsanity.com/paper/2506.19999/) (2025)

## Abstract (arXiv)

Probing has shown that language model representations encode rich linguistic information, but it remains unclear whether they also capture cognitive signals about human processing. In this work, we probe language model representations for human reading times. Using regularized linear regression on two eye-tracking corpora spanning five languages (English, Greek, Hebrew, Russian, and Turkish), we compare the representations from every model layer against scalar predictors -- surprisal, information value, and logit-lens surprisal. We find that the representations from early layers outperform surprisal in predicting early-pass measures such as first fixation and gaze duration. The concentration of predictive power in the early layers suggests that human-like processing signatures are captured by low-level structural or lexical representations, pointing to a functional alignment between model depth and the temporal stages of human reading. In contrast, for late-pass measures such as total reading time, scalar surprisal remains superior, despite its being a much more compressed representation. We also observe performance gains when using both surprisal and early-layer representations. Overall, we find that the best-performing predictor varies strongly depending on the language and eye-tracking measure.