STRUCTURALCOST: A controlled reading time dataset for modeling human sentence processing difficulty
Not on the map. Why?
Abstract (arXiv)
We introduce STRUCTURALCOST, a self-paced reading dataset of 475 participants and 40,800 observations isolating the processing cost of long-distance subject-verb dependency resolution. We replicate a low-powered psycholinguistic finding at NLP scale, namely that human reading times at the main verb increase with dependency length, driven by syntactic embedding beyond linear distance. Different language models -- spanning n-gram models, SSMs, and transformers -- partially mirror this graded difficulty profile, yet underestimate the integration cost humans incur, with a gap that persists across architectures and model sizes. This suggests these models capture the predictive component of human processing but not the full integration cost that working memory imposes. STRUCTURALCOST provides data needed to drive progress toward evaluating the cognitive plausibility of language models.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). First posted on arXiv 6 Oct 2026.
Builds on
No reference list yet.
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Dual Alignment Between Language Model Layers and Human Sentence Processing
- 2023Temperature-scaling surprisal estimates improve fit to human reading times -- but does it do so for the "right reasons"?
- 2026Timesteps of Mamba Align with Human Reading Times
- 2023Can training neural language models on a curriculum with developmentally plausible data improve alignment with human reading behavior?
- 2025Adding LLMs to the psycholinguistic norming toolbox: A practical guide to getting the most out of human ratings
- 2022Training language models to summarize narratives improves brain alignment
- 2023Large Linguistic Models: Investigating LLMs' metalinguistic abilities
- 2020Language Models are Few-Shot Learners