# So Cloze yet so Far: N400 Amplitude is Better Predicted by Distributional Information than Human Predictability Judgements

arXiv: https://arxiv.org/abs/2109.01226 · HTML: https://mlsanity.com/paper/2109.01226/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 97 references that are in the map.
- First posted on arXiv 2 Sep 2021.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Distributed Representations of Words and Phrases and their Compositionality](https://mlsanity.com/paper/1310.4546/) (2013)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [A Primer in BERTology: What we know about how BERT works](https://mlsanity.com/paper/2002.12327/) (2020)
- [Colorless green recurrent networks dream hierarchically](https://mlsanity.com/paper/1803.11138/) (2018)
- [Exploring the Limits of Language Modeling](https://mlsanity.com/paper/1602.02410/) (2016)
- [Neural Language Models as Psycholinguistic Subjects: Representations of Syntactic State](https://mlsanity.com/paper/1903.03260/) (2019)
- [Human Sentence Processing: Recurrence or Attention?](https://mlsanity.com/paper/2005.09471/) (2020)
- [How well does surprisal explain N400 amplitude under different experimental conditions?](https://mlsanity.com/paper/2010.04844/) (2020)
- [Different kinds of cognitive plausibility: why are transformers better than RNNs at predicting N400 amplitude?](https://mlsanity.com/paper/2107.09648/) (2021)

## Built on by (most cited first, 15 of 15)

- [Do Large Language Models know what humans know?](https://mlsanity.com/paper/2209.01515/) (2022)
- [Structural Priming Demonstrates Abstract Grammatical Representations in Multilingual Language Models](https://mlsanity.com/paper/2311.09194/) (2023)
- [When Language Models Fall in Love: Animacy Processing in Transformer Language Models](https://mlsanity.com/paper/2310.15004/) (2023)
- [Can Peanuts Fall in Love with Distributional Semantics?](https://mlsanity.com/paper/2301.08731/) (2023)
- [Revenge of the Fallen? Recurrent Models Match Transformers at Predicting Human Language Comprehension Metrics](https://mlsanity.com/paper/2404.19178/) (2024)
- [Collateral facilitation in humans and language models](https://mlsanity.com/paper/2211.05198/) (2022)
- [Rarely a problem? Language models exhibit inverse scaling in their predictions following few-type quantifiers](https://mlsanity.com/paper/2212.08700/) (2022)
- [Decomposition of surprisal: Unified computational model of ERP components in language processing](https://mlsanity.com/paper/2409.06803/) (2024)
- [N-gram-like Language Models Predict Naturalistic Reading Time Best](https://mlsanity.com/paper/2603.09872/) (2026)
- [Clozing the Gap: Exploring Why Language Model Surprisal Outperforms Cloze Surprisal](https://mlsanity.com/paper/2601.09886/) (2026)
- [A unified information-theoretic model of EEG signatures of human language processing](https://mlsanity.com/paper/2212.08205/) (2022)
- [Learning to Write Rationally: How Information Is Distributed in Non-Native Speakers' Essays](https://mlsanity.com/paper/2411.03550/) (2024)
- [Detecting Reading-Induced Confusion Using EEG and Eye Tracking](https://mlsanity.com/paper/2508.14442/) (2025)
- [How do Humans Process AI-generated Hallucination Contents: a Neuroimaging Study](https://mlsanity.com/paper/2605.16953/) (2026)
- [Decoding EEG Signals to Explore Next-Word Predictability in the Human Brain](https://mlsanity.com/paper/2607.18321/) (2026)

## Abstract (arXiv)

More predictable words are easier to process - they are read faster and elicit smaller neural signals associated with processing difficulty, most notably, the N400 component of the event-related brain potential. Thus, it has been argued that prediction of upcoming words is a key component of language comprehension, and that studying the amplitude of the N400 is a valuable way to investigate the predictions we make. In this study, we investigate whether the linguistic predictions of computational language models or humans better reflect the way in which natural language stimuli modulate the amplitude of the N400. One important difference in the linguistic predictions of humans versus computational language models is that while language models base their predictions exclusively on the preceding linguistic context, humans may rely on other factors. We find that the predictions of three top-of-the-line contemporary language models - GPT-3, RoBERTa, and ALBERT - match the N400 more closely than human predictions. This suggests that the predictive processes underlying the N400 may be more sensitive to the surface-level statistics of language than previously thought.