mlsanityData to 13 Oct 2026

Different kinds of cognitive plausibility: why are transformers better than RNNs at predicting N400 amplitude?

arXiv 2107.09648 ↗First posted 20 Jul 2021cs.CL cs.AI cs.LGcited by 14

Not on the map. Why?

Abstract (arXiv)

Despite being designed for performance rather than cognitive plausibility, transformer language models have been found to be better at predicting metrics used to assess human language comprehension than language models with other architectures, such as recurrent neural networks. Based on how well they predict the N400, a neural signal associated with processing difficulty, we propose and provide evidence for one possible explanation - their predictions are affected by the preceding context in a way analogous to the effect of semantic facilitation in humans.

Key facts. Cited by 14 papers in the map (data through 13 Oct 2026). Builds on 6 of its 41 references that are in the map. First posted on arXiv 20 Jul 2021.

Builds on

6 of 41 references.

References · 6

  1. 2017Attention Is All You Needcited by 60,106
  2. 2019What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language modelscited by 317
  3. 2016Exploring the Limits of Language Modelingcited by 172
  4. 2020On the Predictive Power of Neural Language Models for Human Real-Time Comprehension Behaviorcited by 118
  5. 2020Exploring BERT's Sensitivity to Lexical Cues using Tests from Semantic Primingcited by 41
  6. 2020How well does surprisal explain N400 amplitude under different experimental conditions?cited by 18

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2021: 1; 2022: 4; 2023: 4; 2024: 2; 2025: 2; 2026: 1.

14 papers.

Sort:

  1. 2022Context Limitations Make Neural Language Models More Human-Likecited by 33
  2. 2021So Cloze yet so Far: N400 Amplitude is Better Predicted by Distributional Information than Human Predictability Judgementscited by 15
  3. 2025Language Model Behavioral Phases are Consistent Across Architecture, Training Data, and Scalecited by 13
  4. 2023When Language Models Fall in Love: Animacy Processing in Transformer Language Modelscited by 12
  5. 2022Testing the limits of natural language models for predicting human language judgmentscited by 7
  6. 2023Can Peanuts Fall in Love with Distributional Semantics?cited by 6
  7. 2024Revenge of the Fallen? Recurrent Models Match Transformers at Predicting Human Language Comprehension Metricscited by 6
  8. 2022Collateral facilitation in humans and language modelscited by 5
  9. 2022Rarely a problem? Language models exhibit inverse scaling in their predictions following few-type quantifierscited by 4
  10. 2023Psychometric Predictive Power of Large Language Modelscited by 4
  11. 2025If Attention Serves as a Cognitive Model of Human Memory Retrieval, What is the Plausible Memory Representation?cited by 4
  12. 2024Decomposition of surprisal: Unified computational model of ERP components in language processingcited by 2
2 more
  1. 2026N-gram-like Language Models Predict Naturalistic Reading Time Bestcited by 2
  2. 2023Are words equally surprising in audio and audio-visual comprehension?cited by 1