mlsanityData to 7 Oct 2026

Construction Grammar Provides Unique Insight into Neural Language Models

arXiv 2302.02178 ↗First posted 4 Feb 2023cs.CLcited by 13

Not on the map. Why?

Abstract (arXiv)

Construction Grammar (CxG) has recently been used as the basis for probing studies that have investigated the performance of large pretrained language models (PLMs) with respect to the structure and meaning of constructions. In this position paper, we make suggestions for the continuation and augmentation of this line of research. We look at probing methodology that was not designed with CxG in mind, as well as probing methodology that was designed for specific constructions. We analyse selected previous work in detail, and provide our view of the most important challenges and research questions that this promising new field faces.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 14 of its 42 references that are in the map. First posted on arXiv 4 Feb 2023.

Builds on

14 of 42 references.

References · 14

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  3. 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,970
  4. 2020A Primer in BERTology: What we know about how BERT workscited by 807
  5. 2019Visualizing and Measuring the Geometry of BERTcited by 215
  6. 2019Controllable Sentence Simplificationcited by 78
  7. 2021Frequency Effects on Syntactic Rule Learning in Transformerscited by 59
  8. 2019Does BERT Make Any Sense? Interpretable Word Sense Disambiguation with Contextualized Embeddingscited by 55
  9. 2022The Better Your Syntax, the Better Your Semantics? Probing Pretrained Language Models for the English Comparative Correlativecited by 35
  10. 2020CxGBERT: BERT meets Construction Grammarcited by 32
  11. 2022Neural reality of argument structure constructionscited by 25
  12. 2019Probing for Semantic Classes: Diagnosing the Meaning Content of Word Embeddingscited by 15
2 more
  1. 2021Putting Words in BERT's Mouth: Navigating Contextualized Vector Spaces with Pseudowordscited by 9
  2. 2022Are Representations Built from the Ground Up? An Empirical Examination of Local Composition in Language Modelscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 3; 2025: 4; 2026: 3.

13 papers.

Sort:

  1. 2023Can Language Models Be Tricked by Language Illusions? Easier with Syntax, Harder with Semanticscited by 11
  2. 2025Do Construction Distributions Shape Formal Language Learning In German BabyLMs?cited by 11
  3. 2025Linguistic Generalizations are not Rules: Impacts on Evaluation of LMscited by 10
  4. 2024UCxn: Typologically Informed Annotation of Constructions Atop Universal Dependenciescited by 7
  5. 2025Constructions are Revealed in Word Distributionscited by 6
  6. 2024Analysis of Argument Structure Constructions in the Large Language Model BERTcited by 3
  7. 2023Enhancing Language Representation with Constructional Information for Natural Language Understandingcited by 2
  8. 2023Adverbs, Surprisinglycited by 1
  9. 2024The Computational Learning of Construction Grammars: State of the Art and Prospective Roadmapcited by 1
  10. 2025LLMs Learn Constructions That Humans Do Not Knowcited by 1
  11. 2026Developmental approach reveals the statistical learning of Neural Language Models: Transformers generalize from the most abstract statistical patternscited by 1
  12. 2026Language Models Learn Constructional Semantics, Not To Mention Syntax: Investigating LM Understanding of Paired-Focus Constructionscited by 0
1 more
  1. 2026Neural Language Models Learn the Contextual Distributions of Dependency Structures: a statistical learning theory to compositionalitycited by 0