mlsanityData to 14 Oct 2026

Revisiting the Estimation of Unigram Surprisal in Computational Psycholinguistics

arXiv 2610.18414 ↗First posted 13 Oct 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Contextual and unigram surprisal are standard predictors of reading time in computational psycholinguistics. While contextual surprisal is typically derived from neural language models, unigram surprisal is usually obtained from independent corpus counts. Although both quantities are deterministic functionals of a single underlying distribution---capturing conditional and marginal structure respectively---this practice breaks their probabilistic link and may introduce estimation error from the mismatch. We propose three principles for estimating unigram surprisal in psycholinguistic research, including the requirement that both quantities be derived from the same language model. We operationalize this principle by introducing a Monte Carlo estimator of unigram surprisal from a language model, along with a marginalized variant that has lower variance empirically. When unigram surprisal is re-estimated under this unified framework, we find evidence that predictors change in their predictive contribution to reading time. We also confirm prior findings that frequency can be a stronger predictor of processing effort than contextual surprisal under this unified framework.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 18 of its 75 references that are in the map. First posted on arXiv 13 Oct 2026.

Builds on

18 of 75 references.

References · 18

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2024OLMo: Accelerating the Science of Language Modelscited by 613
  3. 2020On the Predictive Power of Neural Language Models for Human Real-Time Comprehension Behaviorcited by 119
  4. 2022Why Does Surprisal From Larger Transformer-Based Language Models Provide a Poorer Fit to Human Reading Times?cited by 113
  5. 2022What Artificial Neural Networks Can Tell Us About Human Language Acquisitioncited by 99
  6. 2023Testing the Predictions of Surprisal Theory in 11 Languagescited by 77
  7. 2023Transformer-Based Language Model Surprisal Predicts Human Reading Times Best with About Two Billion Training Tokenscited by 41
  8. 2024Frequency Explains the Inverse Correlation of Large Language Models' Size, Training Data Amount, and Surprisal's Fit to Reading Timescited by 39
  9. 2022Context Limitations Make Neural Language Models More Human-Likecited by 34
  10. 2024From Language Models over Tokens to Language Models over Characterscited by 31
  11. 2024Generalized Measures of Anticipation and Responsivity in Online Language Processingcited by 24
  12. 2024On the Role of Context in Reading Time Predictioncited by 20
6 more
  1. 2022Estimating the Entropy of Linguistic Distributionscited by 12
  2. 2026Transducing Language Modelscited by 5
  3. 2025How Well Does First-Token Entropy Approximate Word Entropy as a Psycholinguistic Predictor?cited by 4
  4. 2025A Spatio-Temporal Point Process for Fine-Grained Modeling of Reading Behaviorcited by 3
  5. 2024Reverse-Engineering the Readercited by 2
  6. 2026The Frequency Confound in Language-Model Surprisal and Metaphor Noveltycited by 1

Built on by

Not cited yet.