# Neural reality of argument structure constructions

arXiv: https://arxiv.org/abs/2202.12246 · HTML: https://mlsanity.com/paper/2202.12246/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 63 references that are in the map.
- First posted on arXiv 24 Feb 2022.

## Builds on (most cited first, 27 of 27)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Neural Network Acceptability Judgments](https://mlsanity.com/paper/1805.12471/) (2018)
- [How Contextual are Contextualized Word Representations? Comparing the Geometry of BERT, ELMo, and GPT-2 Embeddings](https://mlsanity.com/paper/1909.00512/) (2019)
- [BLiMP: The Benchmark of Linguistic Minimal Pairs for English](https://mlsanity.com/paper/1912.00582/) (2019)
- [Linguistic Knowledge and Transferability of Contextual Representations](https://mlsanity.com/paper/1903.08855/) (2019)
- [Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies](https://mlsanity.com/paper/1611.01368/) (2016)
- [What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language models](https://mlsanity.com/paper/1907.13528/) (2019)
- [Colorless green recurrent networks dream hierarchically](https://mlsanity.com/paper/1803.11138/) (2018)
- [A Systematic Assessment of Syntactic Generalization in Neural Language Models](https://mlsanity.com/paper/2005.03692/) (2020)
- [All Bark and No Bite: Rogue Dimensions in Transformer Language Models Obscure Representational Quality](https://mlsanity.com/paper/2109.04404/) (2021)
- [BERT Busters: Outlier Dimensions that Disrupt Transformers](https://mlsanity.com/paper/2105.06990/) (2021)
- [When Do You Need Billions of Words of Pretraining Data?](https://mlsanity.com/paper/2011.04946/) (2020)
- [Syntactic Structure from Deep Learning](https://mlsanity.com/paper/2004.10827/) (2020)
- [Out of Order: How Important Is The Sequential Order of Words in a Sentence in Natural Language Understanding Tasks?](https://mlsanity.com/paper/2012.15180/) (2020)
- [Probing Across Time: What Does RoBERTa Know and When?](https://mlsanity.com/paper/2104.07885/) (2021)
- [UnNatural Language Inference](https://mlsanity.com/paper/2101.00010/) (2020)
- [Frequency Effects on Syntactic Rule Learning in Transformers](https://mlsanity.com/paper/2109.07020/) (2021)
- [Exploring BERT's Sensitivity to Lexical Cues using Tests from Semantic Priming](https://mlsanity.com/paper/2010.03010/) (2020)
- [Assessing Phrasal Representation and Composition in Transformers](https://mlsanity.com/paper/2010.03763/) (2020)
- [CxGBERT: BERT meets Construction Grammar](https://mlsanity.com/paper/2011.04134/) (2020)
- [A Targeted Assessment of Incremental Processing in Neural LanguageModels and Humans](https://mlsanity.com/paper/2106.03232/) (2021)
- [Using Priming to Uncover the Organization of Syntactic Representations in Neural Language Models](https://mlsanity.com/paper/1909.10579/) (2019)
- [Spying on your neighbors: Fine-grained probing of contextual embeddings for information about surrounding words](https://mlsanity.com/paper/2005.01810/) (2020)
- [How is BERT surprised? Layerwise detection of linguistic anomalies](https://mlsanity.com/paper/2105.07452/) (2021)
- [How well does surprisal explain N400 amplitude under different experimental conditions?](https://mlsanity.com/paper/2010.04844/) (2020)
- [Learning Which Features Matter: RoBERTa Acquires a Preference for Linguistic Generalizations (Eventually)](https://mlsanity.com/paper/2010.05358/) (2020)

## Built on by (most cited first, 25 of 25)

- [Language Models Learn Rare Phenomena from Less Rare Phenomena: The Case of the Missing AANNs](https://mlsanity.com/paper/2403.19827/) (2024)
- [The Better Your Syntax, the Better Your Semantics? Probing Pretrained Language Models for the English Comparative Correlative](https://mlsanity.com/paper/2210.13181/) (2022)
- [A Discerning Several Thousand Judgments: GPT-3 Rates the Article + Adjective + Numeral + Noun Construction](https://mlsanity.com/paper/2301.12564/) (2023)
- [Structural Priming Demonstrates Abstract Grammatical Representations in Multilingual Language Models](https://mlsanity.com/paper/2311.09194/) (2023)
- [Predicting Fine-Tuning Performance with Probing](https://mlsanity.com/paper/2210.07352/) (2022)
- [Construction Grammar Provides Unique Insight into Neural Language Models](https://mlsanity.com/paper/2302.02178/) (2023)
- [Linguistic Generalizations are not Rules: Impacts on Evaluation of LMs](https://mlsanity.com/paper/2502.13195/) (2025)
- [On the Acquisition of Shared Grammatical Representations in Bilingual Language Models](https://mlsanity.com/paper/2503.03962/) (2025)
- [Constructions are Revealed in Word Distributions](https://mlsanity.com/paper/2503.06048/) (2025)
- [Hybrid Human-LLM Corpus Construction and LLM Evaluation for Rare Linguistic Phenomena](https://mlsanity.com/paper/2403.06965/) (2024)
- [Beyond Memorization: Assessing Semantic Generalization in Large Language Models Using Phrasal Constructions](https://mlsanity.com/paper/2501.04661/) (2025)
- [Construction Identification and Disambiguation Using BERT: A Case Study of NPN](https://mlsanity.com/paper/2503.18751/) (2025)
- [Unpacking Let Alone: Human-Scale Models Generalize to a Rare Construction in Form but not Meaning](https://mlsanity.com/paper/2506.04408/) (2025)
- [Syntax and Semantics Meet in the "Middle": Probing the Syntax-Semantics Interface of LMs Through Agentivity](https://mlsanity.com/paper/2305.18185/) (2023)
- [Analysis of Argument Structure Constructions in the Large Language Model BERT](https://mlsanity.com/paper/2408.04270/) (2024)
- [Construction Grammar and Language Models](https://mlsanity.com/paper/2308.13315/) (2023)
- [LLMs Learn Constructions That Humans Do Not Know](https://mlsanity.com/paper/2508.16837/) (2025)
- [Uniform Information Density and Syntactic Reduction: Revisiting textitthat-Mentioning in English Complement Clauses](https://mlsanity.com/paper/2509.05254/) (2025)
- [Leveraging Lora Fine-Tuning and Knowledge Bases for Construction Identification](https://mlsanity.com/paper/2601.13105/) (2026)
- [Do Language Models Know What Not to Say? Causal Evidence for Statistical Preemption in LLMs](https://mlsanity.com/paper/2605.23039/) (2026)
- [Developmental approach reveals the statistical learning of Neural Language Models: Transformers generalize from the most abstract statistical patterns](https://mlsanity.com/paper/2606.27460/) (2026)
- [Crosslingual Structural Priming and the Pre-Training Dynamics of Bilingual Language Models](https://mlsanity.com/paper/2310.07929/) (2023)
- [FinGen: A Dataset for Argument Generation in Finance](https://mlsanity.com/paper/2405.20708/) (2024)
- [A Hierarchical and Attentional Analysis of Argument Structure Constructions in BERT Using Naturalistic Corpora](https://mlsanity.com/paper/2602.00554/) (2026)
- [Language Models Learn Constructional Semantics, Not To Mention Syntax: Investigating LM Understanding of Paired-Focus Constructions](https://mlsanity.com/paper/2605.31586/) (2026)

## Abstract (arXiv)

In lexicalist linguistic theories, argument structure is assumed to be predictable from the meaning of verbs. As a result, the verb is the primary determinant of the meaning of a clause. In contrast, construction grammarians propose that argument structure is encoded in constructions (or form-meaning pairs) that are distinct from verbs. Decades of psycholinguistic research have produced substantial empirical evidence in favor of the construction view. Here we adapt several psycholinguistic studies to probe for the existence of argument structure constructions (ASCs) in Transformer-based language models (LMs). First, using a sentence sorting experiment, we find that sentences sharing the same construction are closer in embedding space than sentences sharing the same verb. Furthermore, LMs increasingly prefer grouping by construction with more input data, mirroring the behaviour of non-native language learners. Second, in a "Jabberwocky" priming-based experiment, we find that LMs associate ASCs with meaning, even in semantically nonsensical sentences. Our work offers the first evidence for ASCs in LMs and highlights the potential to devise novel probing methods grounded in psycholinguistic research.