mlsanityData to 7 Oct 2026

Collateral facilitation in humans and language models

arXiv 2211.05198 ↗First posted 9 Nov 2022cs.CL cs.AIcited by 5

Not on the map. Why?

Abstract (arXiv)

Are the predictions of humans and language models affected by similar things? Research suggests that while comprehending language, humans make predictions about upcoming words, with more predictable words being processed more easily. However, evidence also shows that humans display a similar processing advantage for highly anomalous words when these words are semantically related to the preceding context or to the most probable continuation. Using stimuli from 3 psycholinguistic experiments, we find that this is also almost always also the case for 8 contemporary transformer language models (BERT, ALBERT, RoBERTa, XLM-R, GPT-2, GPT-Neo, GPT-J, and XGLM). We then discuss the implications of this phenomenon for our understanding of both human language comprehension and the predictions made by language models.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 21 of its 90 references that are in the map. First posted on arXiv 9 Nov 2022.

Builds on

21 of 90 references.

References · 21

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  4. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  5. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,907
  6. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,464
  7. 2019ALBERT: A Lite BERT for Self-supervised Learning of Language Representationscited by 2,772
  8. 2022LaMDA: Language Models for Dialog Applicationscited by 1,218
  9. 2019What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language modelscited by 317
  10. 2019Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Flycited by 240
  11. 2021CANINE: Pre-training an Efficient Tokenization-Free Encoder for Language Representationcited by 229
  12. 2018Colorless green recurrent networks dream hierarchicallycited by 210
9 more
  1. 2016Exploring the Limits of Language Modelingcited by 172
  2. 2019Neural Language Models as Psycholinguistic Subjects: Representations of Syntactic Statecited by 137
  3. 2015Character-Aware Neural Language Modelscited by 129
  4. 2020Human Sentence Processing: Recurrence or Attention?cited by 51
  5. 2020Exploring BERT's Sensitivity to Lexical Cues using Tests from Semantic Primingcited by 41
  6. 2019Using Priming to Uncover the Organization of Syntactic Representations in Neural Language Modelscited by 29
  7. 2020How well does surprisal explain N400 amplitude under different experimental conditions?cited by 18
  8. 2021So Cloze yet so Far: N400 Amplitude is Better Predicted by Distributional Information than Human Predictability Judgementscited by 15
  9. 2021Different kinds of cognitive plausibility: why are transformers better than RNNs at predicting N400 amplitude?cited by 14

Built on by

Cited by year

Citing papers per year: 2023: 3; 2024: 1; 2025: 1.

5 papers.

  1. 2023Do LLMs exhibit human-like response biases? A case study in survey designcited by 176
  2. 2023Language Model Behavior: A Comprehensive Surveycited by 92
  3. 2025Language Model Behavioral Phases are Consistent Across Architecture, Training Data, and Scalecited by 13
  4. 2023Can Peanuts Fall in Love with Distributional Semantics?cited by 6
  5. 2024Revenge of the Fallen? Recurrent Models Match Transformers at Predicting Human Language Comprehension Metricscited by 6