mlsanityData to 7 Oct 2026

Human Sentence Processing: Recurrence or Attention?

arXiv 2005.09471 ↗First posted 19 May 2020cs.CLcited by 51

Not on the map. Why?

Abstract (arXiv)

Recurrent neural networks (RNNs) have long been an architecture of interest for computational models of human sentence processing. The recently introduced Transformer architecture outperforms RNNs on many natural language processing tasks but little is known about its ability to model human language processing. We compare Transformer- and RNN-based language models' ability to account for measures of human reading effort. Our analysis shows Transformers to outperform RNNs in explaining self-paced reading times and neural activity during reading English sentences, challenging the widely held idea that human sentence processing involves recurrent and immediate processing and provides evidence for cue-based retrieval.

Key facts. Cited by 51 papers in the map (data through 7 Oct 2026). First posted on arXiv 19 May 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 5; 2022: 15; 2023: 11; 2024: 10; 2025: 7; 2026: 3.

51 papers.

Sort:

By area:

  1. 2022Why Does Surprisal From Larger Transformer-Based Language Models Provide a Poorer Fit to Human Reading Times?cited by 111
  2. 2022minicons: Enabling Flexible Behavioral and Representational Analyses of Transformer Language Modelscited by 58
  3. 2021Lower Perplexity is Not Always Human-Likecited by 52
  4. 2021Incorporating Residual and Normalization Layers into Analysis of Masked Language Modelscited by 50
  5. 2023Transformer-Based Language Model Surprisal Predicts Human Reading Times Best with About Two Billion Training Tokenscited by 39
  6. 2022Context Limitations Make Neural Language Models More Human-Likecited by 32
  7. 2025Large Language Models Are Human-Like Internallycited by 28
  8. 2024Generalized Measures of Anticipation and Responsivity in Online Language Processingcited by 23
  9. 2023Eyettention: An Attention-based Dual-Sequence Model for Predicting Human Scanpaths during Readingcited by 18
  10. 2021So Cloze yet so Far: N400 Amplitude is Better Predicted by Distributional Information than Human Predictability Judgementscited by 15
  11. 2022Entropy- and Distance-Based Predictors From GPT-2 Attention Patterns Predict Reading Times Over and Above GPT-2 Surprisalcited by 14
  12. 2024Linear Recency Bias During Training Improves Transformers' Fit to Reading Timescited by 14
38 more
  1. 2024A Psycholinguistic Evaluation of Language Models' Sensitivity to Argument Rolescited by 14
  2. 2023ScanDL: A Diffusion Model for Generating Synthetic Scanpaths on Textscited by 13
  3. 2023When Language Models Fall in Love: Animacy Processing in Transformer Language Modelscited by 12
  4. 2024On the Proper Treatment of Tokenization in Psycholinguisticscited by 12
  5. 2025To model human linguistic prediction, make LLMs less superhumancited by 11
  6. 2022Eye-tracking based classification of Mandarin Chinese readers with and without dyslexia using neural sequence modelscited by 10
  7. 2025ConfPO: Exploiting Policy Model Confidence for Critical Token Selection in Preference Optimizationcited by 10
  8. 2023Expectations over Unspoken Alternatives Predict Pragmatic Inferencescited by 9
  9. 2021Can Transformer Language Models Predict Psychometric Properties?cited by 8
  10. 2022Measuring the Impact of (Psycho-)Linguistic and Readability Features and Their Spill Over Effects on the Prediction of Eye Movement Patternscited by 8
  11. 2022Continuous-Time Audiovisual Fusion with Recurrence vs. Attention for In-The-Wild Affect Recognitioncited by 7
  12. 2022Testing the limits of natural language models for predicting human language judgmentscited by 7
  13. 2023Roles of Scaling and Instruction Tuning in Language Perception: Model vs. Human Attentioncited by 7
  14. 2023Temperature-scaling surprisal estimates improve fit to human reading times -- but does it do so for the "right reasons"?cited by 7
  15. 2023Can Peanuts Fall in Love with Distributional Semantics?cited by 6
  16. 2024Revenge of the Fallen? Recurrent Models Match Transformers at Predicting Human Language Comprehension Metricscited by 6
  17. 2021Modeling Human Sentence Processing with Left-Corner Recurrent Neural Network Grammarscited by 5
  18. 2022Predicting Human Psychometric Properties Using Computational Language Modelscited by 5
  19. 2022Construction Repetition Reduces Information Rate in Dialoguecited by 5
  20. 2022Collateral facilitation in humans and language modelscited by 5
  21. 2024EMTeC: A Corpus of Eye Movements on Machine-Generated Textscited by 5
  22. 2024The Impact of Token Granularity on the Predictive Power of Language Model Surprisalcited by 5
  23. 2022Rarely a problem? Language models exhibit inverse scaling in their predictions following few-type quantifierscited by 4
  24. 2023Psychometric Predictive Power of Large Language Modelscited by 4
  25. 2024Computational Models to Study Language Processing in the Human Brain: A Surveycited by 4
  26. 2025If Attention Serves as a Cognitive Model of Human Memory Retrieval, What is the Plausible Memory Representation?cited by 4
  27. 2024Modeling Bilingual Sentence Processing: Evaluating RNN and Transformer Architectures for Cross-Language Structural Primingcited by 3
  28. 2022A Comprehensive Comparison of Neural Networks as Cognitive Models of Inflectioncited by 2
  29. 2024Decomposition of surprisal: Unified computational model of ERP components in language processingcited by 2
  30. 2022Do language models make human-like predictions about the coreferents of Italian anaphoric zero pronouns?cited by 1
  31. 2023Are words equally surprising in audio and audio-visual comprehension?cited by 1
  32. 2023Plausibility Processing in Transformer Language Models: Focusing on the Role of Attention Heads in GPTcited by 1
  33. 2025Surprisal from Larger Transformer-based Language Models Predicts fMRI Data More Poorlycited by 1
  34. 2025Topoformer: brain-like topographic organization in Transformer language models through spatial querying and reweightingcited by 1
  35. 2026N-gram-like Language Models Predict Naturalistic Reading Time Bestcited by 1
  36. 2022Text Difficulty Study: Do machines behave the same as humans regarding text difficulty?cited by 0
  37. 2025Do Large Language Models know who did what to whom?cited by 0
  38. 2026Quantifying the cross-linguistic effects of syncretism on agreement attractioncited by 0

50 most cited of 51 All 51 (JSON)