Dual Alignment Between Language Model Layers and Human Sentence Processing
Not on the map. Why?
Abstract (arXiv)
A recent study (Kuribayashi et al., 2025) has shown that human sentence processing behavior, typically measured on syntactically unchallenging constructions, can be effectively modeled using surprisal from early layers of large language models (LLMs). This raises the question of whether such advantages of internal layers extend to more syntactically challenging constructions, where surprisal has been reported to underestimate human cognitive effort. In this paper, we begin by exploring internal layers that better estimate human cognitive effort observed in syntactic ambiguity processing in English. Our experiments show that, in contrast to naturalistic reading, later layers better estimate such a cognitive effort, but still underestimate the human data. This dual alignment sheds light on different modes of sentence processing in humans and LMs: naturalistic reading employs a somewhat weak prediction akin to earlier layers of LMs, while syntactically challenging processing requires more fully-contextualized representations, better modeled by later layers of LMs. Motivated by these findings, we also explore several probability-update measures using shallow and deep layers of LMs, showing a complementary advantage to single-layer's surprisal in reading time modeling.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 15 of its 54 references that are in the map. First posted on arXiv 20 Apr 2026.
Builds on
15 of 54 references.
References · 15
- 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
- 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,046
- 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579
- 2020On the Predictive Power of Neural Language Models for Human Real-Time Comprehension Behaviorcited by 115
- 2022Why Does Surprisal From Larger Transformer-Based Language Models Provide a Poorer Fit to Human Reading Times?cited by 111
- 2021Lower Perplexity is Not Always Human-Likecited by 52
- 2018Finding Syntax in Human Encephalography with Beam Searchcited by 41
- 2021A Targeted Assessment of Incremental Processing in Neural LanguageModels and Humanscited by 32
- 2022Context Limitations Make Neural Language Models More Human-Likecited by 32
- 2022Syntactic Surprisal From Neural Models Predicts, But Underestimates, Human Processing Difficulty From Syntactic Ambiguitiescited by 30
- 2024Leading Whitespaces of Language Models' Subword Vocabulary Pose a Confound for Calculating Word Probabilitiescited by 30
- 2025Large Language Models Are Human-Like Internallycited by 28
3 more
- 2024Decoding Probing: Revealing Internal Linguistic Structures in Neural Language Models using Minimal Pairscited by 24
- 2024An information-theoretic model of shallow and deep language comprehensioncited by 6
- 2024The Impact of Token Granularity on the Predictive Power of Language Model Surprisalcited by 5
Built on by
Citing papers per year: 2026: 2.
2 papers.
Similar, not cited
Close in wording, no citation either way
- 2026STRUCTURALCOST: A controlled reading time dataset for modeling human sentence processing difficulty
- 2025Fine-grained Analysis of Brain-LLM Alignment through Input Attribution
- 2025Do Large Language Models Think Like the Brain? Sentence-Level Evidences from Layer-Wise Embeddings and fMRI
- 2023Can training neural language models on a curriculum with developmentally plausible data improve alignment with human reading behavior?
- 2023Temperature-scaling surprisal estimates improve fit to human reading times -- but does it do so for the "right reasons"?
- 2022Training language models to summarize narratives improves brain alignment
- 2023Instruction-tuning Aligns LLMs to the Human Brain
- 2026Linguistic properties and model scale in brain encoding: from small to compressed language models