mlsanityData to 14 Oct 2026

Probing for Reading Times

arXiv 2604.18712 ↗First posted 20 Apr 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Probing has shown that language model representations encode rich linguistic information, but it remains unclear whether they also capture cognitive signals about human processing. In this work, we probe language model representations for human reading times. Using regularized linear regression on two eye-tracking corpora spanning five languages (English, Greek, Hebrew, Russian, and Turkish), we compare the representations from every model layer against scalar predictors -- surprisal, information value, and logit-lens surprisal. We find that the representations from early layers outperform surprisal in predicting early-pass measures such as first fixation and gaze duration. The concentration of predictive power in the early layers suggests that human-like processing signatures are captured by low-level structural or lexical representations, pointing to a functional alignment between model depth and the temporal stages of human reading. In contrast, for late-pass measures such as total reading time, scalar surprisal remains superior, despite its being a much more compressed representation. We also observe performance gains when using both surprisal and early-layer representations. Overall, we find that the best-performing predictor varies strongly depending on the language and eye-tracking measure.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 22 of its 47 references that are in the map. First posted on arXiv 20 Apr 2026.

Builds on

22 of 47 references.

References · 22

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  4. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 623
  5. 2024Do Llamas Work in English? On the Latent Language of Multilingual Transformerscited by 281
  6. 2020On the Predictive Power of Neural Language Models for Human Real-Time Comprehension Behaviorcited by 119
  7. 2022Why Does Surprisal From Larger Transformer-Based Language Models Provide a Poorer Fit to Human Reading Times?cited by 113
  8. 2023Testing the Predictions of Surprisal Theory in 11 Languagescited by 77
  9. 2024Why do language models perform worse for morphologically complex languages?cited by 62
  10. 2024How to Compute the Probability of a Wordcited by 36
  11. 2025Everything, Everywhere, All at Once: Is Mechanistic Interpretability Identifiable?cited by 34
  12. 2021A Non-Linear Structural Probecited by 31
10 more
  1. 2024From Language Models over Tokens to Language Models over Characterscited by 31
  2. 2024Leading Whitespaces of Language Models' Subword Vocabulary Pose a Confound for Calculating Word Probabilitiescited by 30
  3. 2025Large Language Models Are Human-Like Internallycited by 29
  4. 2024Generalized Measures of Anticipation and Responsivity in Online Language Processingcited by 24
  5. 2023Information Value: Measuring Utterance Predictability as Distance from Plausible Alternativescited by 23
  6. 2024The Foundations of Tokenization: Statistical and Computational Concernscited by 22
  7. 2024On the Role of Context in Reading Time Predictioncited by 20
  8. 2022Analyzing Wrap-Up Effects through an Information-Theoretic Lenscited by 10
  9. 2024Introducing cosmosGPT: Monolingual Training for Turkish Language Modelscited by 8
  10. 2025A Spatio-Temporal Point Process for Fine-Grained Modeling of Reading Behaviorcited by 3

Built on by

Not cited yet.