# Do Language Models Exhibit Human-like Structural Priming Effects?

arXiv: https://arxiv.org/abs/2406.04847 · HTML: https://mlsanity.com/paper/2406.04847/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 89 references that are in the map.
- First posted on arXiv 7 Jun 2024.

## Builds on (most cited first, 23 of 23)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [Direct Preference Optimization: Your Language Model is Secretly a Reward Model](https://mlsanity.com/paper/2305.18290/) (2023)
- [Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?](https://mlsanity.com/paper/2202.12837/) (2022)
- [MPNet: Masked and Permuted Pre-training for Language Understanding](https://mlsanity.com/paper/2004.09297/) (2020)
- [The Falcon Series of Open Language Models](https://mlsanity.com/paper/2311.16867/) (2023)
- [Zephyr: Direct Distillation of LM Alignment](https://mlsanity.com/paper/2310.16944/) (2023)
- [Neural Language Models as Psycholinguistic Subjects: Representations of Syntactic State](https://mlsanity.com/paper/1903.03260/) (2019)
- [Syntactic Structure from Deep Learning](https://mlsanity.com/paper/2004.10827/) (2020)
- [Structural Persistence in Language Models: Priming as a Window into Abstract Language Representations](https://mlsanity.com/paper/2109.14989/) (2021)
- [Characterizing Intrinsic Compositionality in Transformers with Tree Projections](https://mlsanity.com/paper/2211.01288/) (2022)
- [Understanding In-Context Learning via Supportive Pretraining Data](https://mlsanity.com/paper/2306.15091/) (2023)
- [Using Priming to Uncover the Organization of Syntactic Representations in Neural Language Models](https://mlsanity.com/paper/1909.10579/) (2019)
- [Investigating representations of verb bias in neural language models](https://mlsanity.com/paper/2010.02375/) (2020)
- [Language model acceptability judgements are not always robust to context](https://mlsanity.com/paper/2212.08979/) (2022)
- [A Neural Model of Adaptation in Reading](https://mlsanity.com/paper/1808.09930/) (2018)
- [Structural Priming Demonstrates Abstract Grammatical Representations in Multilingual Language Models](https://mlsanity.com/paper/2311.09194/) (2023)
- [Transparency at the Source: Evaluating and Interpreting Language Models With Access to the True Distribution](https://mlsanity.com/paper/2310.14840/) (2023)
- [Feature Interactions Reveal Linguistic Structure in Language Models](https://mlsanity.com/paper/2306.12181/) (2023)
- [diagNNose: A Library for Neural Activation Analysis](https://mlsanity.com/paper/2011.06819/) (2020)
- [Attribution and Alignment: Effects of Local Context Repetition on Utterance Production and Comprehension in Dialogue](https://mlsanity.com/paper/2311.13061/) (2023)
- [SPAWNing Structural Priming Predictions from a Cognitively Motivated Parser](https://mlsanity.com/paper/2403.07202/) (2024)
- [Modeling Bilingual Sentence Processing: Evaluating RNN and Transformer Architectures for Cross-Language Structural Priming](https://mlsanity.com/paper/2405.09508/) (2024)

## Built on by (most cited first, 21 of 21)

- [Sketch-of-Thought: Efficient LLM Reasoning with Adaptive Cognitive-Inspired Sketching](https://mlsanity.com/paper/2503.05179/) (2025)
- [Herd Behavior: Investigating Peer Influence in LLM-based Multi-Agent Systems](https://mlsanity.com/paper/2505.21588/) (2025)
- [On the Acquisition of Shared Grammatical Representations in Bilingual Language Models](https://mlsanity.com/paper/2503.03962/) (2025)
- [Strong Memory, Weak Control: An Empirical Study of Executive Functioning in LLMs](https://mlsanity.com/paper/2504.02789/) (2025)
- [Large Language Models as Psychological Simulators: A Methodological Guide](https://mlsanity.com/paper/2506.16702/) (2025)
- [Is In-Context Learning a Type of Error-Driven Learning? Evidence from the Inverse Frequency Effect in Structural Priming](https://mlsanity.com/paper/2406.18501/) (2024)
- [How LLMs Comprehend Temporal Meaning in Narratives: A Case Study in Cognitive Evaluation of LLMs](https://mlsanity.com/paper/2507.14307/) (2025)
- [The Power of Stories: Narrative Priming Shapes How LLM Agents Collaborate and Compete](https://mlsanity.com/paper/2505.03961/) (2025)
- [Social Simulations with Large Language Model Risk Utopian Illusion](https://mlsanity.com/paper/2510.21180/) (2025)
- [A Linguistically Motivated Analysis of Intonational Phrasing in Text-to-Speech Systems: Revealing Gaps in Syntactic Sensitivity](https://mlsanity.com/paper/2505.22236/) (2025)
- [semantic-features: A User-Friendly Tool for Studying Contextual Word Embeddings in Interpretable Semantic Spaces](https://mlsanity.com/paper/2506.06169/) (2025)
- [In-Context Learning in Speech Language Models: Analyzing the Role of Acoustic Features, Linguistic Structure, and Induction Heads](https://mlsanity.com/paper/2604.06356/) (2026)
- [Surprisal Minimisation over Goal-directed Alternatives Predicts Production Choice in Dialogue](https://mlsanity.com/paper/2605.00506/) (2026)
- [Food Noise & False Safety: A Systematic Evaluation of How LLMs Fail to Adapt to Eating Disorder Queries with Clinician Feedback](https://mlsanity.com/paper/2606.02444/) (2026)
- [Animate, or Inanimate, That is the Question for Large Language Models](https://mlsanity.com/paper/2408.06332/) (2024)
- [Towards Human Cognition: Visual Context Guides Syntactic Priming in Fusion-Encoded Models](https://mlsanity.com/paper/2502.17669/) (2025)
- [Is Length Really A Liability? An Evaluation of Multi-turn LLM Conversations using BoolQ](https://mlsanity.com/paper/2601.16508/) (2026)
- [The Effects of Visual Priming on Cooperative Behavior in Vision-Language Models](https://mlsanity.com/paper/2604.27953/) (2026)
- [Analysis of the Neglect-Zero Effect in Large Language Models](https://mlsanity.com/paper/2606.05864/) (2026)
- [Instruction-Tuned Models Locally Reuse Human Syntax More Than Humans Do](https://mlsanity.com/paper/2607.26015/) (2026)
- [Patterns of Priming in Production: Lexical, Semantic and Structural Alignment in Language Model Generation](https://mlsanity.com/paper/2609.04484/) (2026)

## Abstract (arXiv)

We explore which linguistic factors -- at the sentence and token level -- play an important role in influencing language model predictions, and investigate whether these are reflective of results found in humans and human corpora (Gries and Kootstra, 2017). We make use of the structural priming paradigm, where recent exposure to a structure facilitates processing of the same structure. We don't only investigate whether, but also where priming effects occur, and what factors predict them. We show that these effects can be explained via the inverse frequency effect, known in human priming, where rarer elements within a prime increase priming effects, as well as lexical dependence between prime and target. Our results provide an important piece in the puzzle of understanding how properties within their context affect structural prediction in language models.