mlsanityData to 7 Oct 2026

On the scaling relationship between cloze probabilities and language model next-token prediction

arXiv 2602.17848 ↗First posted 19 Feb 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent work has shown that larger language models have better predictive power for eye movement and reading time data. While even the best models under-allocate probability mass to human responses, larger models assign higher-quality estimates of next tokens and their likelihood of production in cloze data because they are less sensitive to lexical co-occurrence statistics while being better aligned semantically to human cloze responses. The results provide support for the claim that the greater memorization capacity of larger models helps them guess more semantically appropriate words, but makes them less sensitive to low-level information that is relevant for word recognition.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 23 of its 54 references that are in the map. First posted on arXiv 19 Feb 2026.

Builds on

23 of 54 references.

References · 23

  1. 2017On Calibration of Modern Neural Networkscited by 4,860
  2. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  3. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,346
  4. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,734
  5. 2021Surface Form Competition: Why the Highest Probability Answer Isn't Always Rightcited by 256
  6. 2022Why Does Surprisal From Larger Transformer-Based Language Models Provide a Poorer Fit to Human Reading Times?cited by 111
  7. 2023What Comes Next? Evaluating Uncertainty in Neural Text Generators Against Human Production Variabilitycited by 51
  8. 2023Transformer-Based Language Model Surprisal Predicts Human Reading Times Best with About Two Billion Training Tokenscited by 39
  9. 2024Frequency Explains the Inverse Correlation of Large Language Models' Size, Training Data Amount, and Surprisal's Fit to Reading Timescited by 37
  10. 2024How to Compute the Probability of a Wordcited by 36
  11. 2024Leading Whitespaces of Language Models' Subword Vocabulary Pose a Confound for Calculating Word Probabilitiescited by 30
  12. 2024Generalized Measures of Anticipation and Responsivity in Online Language Processingcited by 23
11 more
  1. 2022Language model acceptability judgements are not always robust to contextcited by 22
  2. 2023Words, Subwords, and Morphemes: What Really Matters in the Surprisal-Reading Time Relationship?cited by 13
  3. 2025Infini-gram mini: Exact n-gram Search at the Internet Scale with FM-Indexcited by 13
  4. 2025Language Model Behavioral Phases are Consistent Across Architecture, Training Data, and Scalecited by 13
  5. 2023Humans and language models diverge when predicting repeating textcited by 12
  6. 2022Estimating the Entropy of Linguistic Distributionscited by 11
  7. 2024Towards a Similarity-adjusted Surprisal Theorycited by 9
  8. 2024The Effect of Surprisal on Reading Times in Information Seeking and Repeated Readingcited by 8
  9. 2025The Inverse Scaling Effect of Pre-Trained Language Model Surprisal Is Not Due to Data Leakagecited by 3
  10. 2026Clozing the Gap: Exploring Why Language Model Surprisal Outperforms Cloze Surprisalcited by 1
  11. 2026Can LLMs capture stable human-generated sentence entropy measures?cited by 1

Built on by

Not cited yet.