The Grammar of Transformers: A Systematic Review of Interpretability Research on Syntactic Knowledge in Language Models
Not on the map. Why?
Abstract (arXiv)
We present a systematic review of 337 articles evaluating the syntactic abilities of Transformer-based language models (TLMs), reporting on over 3,000 datapoints spanning a wide range of syntactic phenomena, languages, models, and methods. We take the data to collectively show that TLMs encode a non-trivial amount of syntactic knowledge. Behavioral evidence shows strong performance on formal syntactic phenomena, but weaker and more variable performance on phenomena at the syntax-semantics interface. Performance is also consistently lower for languages with less digital support. Probing and mechanistic studies further support the presence of syntactic knowledge in TLMs. Yet, because most work remains observational and methodologically heterogeneous, insight into the detailed computational mechanisms underlying syntactic processing remains limited. At the same time, the literature remains heavily concentrated on English and BERT-like models. We discuss the implications of our results and provide recommendations for future research.
Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 223 of its 354 references that are in the map. First posted on arXiv 9 Jan 2026.
Builds on
223 of 354 references.
References · 223
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
- 2018GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understandingcited by 4,899
- 2018Deep contextualized word representationscited by 2,290
- 2022Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language modelscited by 1,998
- 2019SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systemscited by 1,729
- 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
- 2019BERT Rediscovers the Classical NLP Pipelinecited by 1,059
- 2020Beyond Accuracy: Behavioral Testing of NLP models with CheckListcited by 1,006
- 2019What Does BERT Look At? An Analysis of BERT's Attentioncited by 1,003
- 2023Steering Language Models With Activation Engineeringcited by 922
- 2019Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inferencecited by 876
- 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 876
38 more
- 2020A Primer in BERTology: What we know about how BERT workscited by 815
- 2020The State and Fate of Linguistic Diversity and Inclusion in the NLP Worldcited by 798
- 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
- 2019How multilingual is Multilingual BERT?cited by 645
- 2019On the Cross-lingual Transferability of Monolingual Representationscited by 578
- 2019BLiMP: The Benchmark of Linguistic Minimal Pairs for Englishcited by 564
- 2019What do you learn from context? Probing for sentence structure in contextualized word representationscited by 438
- 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 435
- 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 422
- 2019Linguistic Knowledge and Transferability of Contextual Representationscited by 383
- 2016Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependenciescited by 354
- 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 334
- 2019What BERT is not: Lessons from a new suite of psycholinguistic diagnostics for language modelscited by 320
- 2025Layer by Layer: Uncovering Hidden Representations in Language Modelscited by 311
- 2019Beto, Bentz, Becas: The Surprising Cross-Lingual Effectiveness of BERTcited by 301
- 2019Analyzing the Structure of Attention in a Transformer Language Modelcited by 243
- 2019Visualizing and Measuring the Geometry of BERTcited by 216
- 2018Colorless green recurrent networks dream hierarchicallycited by 213
- 2020Information-Theoretic Probing for Linguistic Structurecited by 207
- 2020Are All Languages Created Equal in Multilingual BERT?cited by 207
- 2019Assessing BERT's Syntactic Abilitiescited by 192
- 2019Cross-Lingual Ability of Multilingual BERT: An Empirical Studycited by 182
- 2021Masked Language Modeling and the Distributional Hypothesis: Order Word Matters Pre-training for Littlecited by 177
- 2020A Systematic Assessment of Syntactic Generalization in Neural Language Modelscited by 165
- 2024The Remarkable Robustness of LLMs: Stages of Inference?cited by 146
- 2021Causal Analysis of Syntactic Agreement Mechanisms in Neural Language Modelscited by 131
- 2020Intermediate-Task Transfer Learning with Pretrained Models for Natural Language Understanding: When and Why Does It Work?cited by 127
- 2023Sudden Drops in the Loss: Syntax Acquisition, Phase Transitions, and Simplicity Bias in MLMscited by 124
- 2019Open Sesame: Getting Inside BERT's Linguistic Knowledgecited by 120
- 2022Should You Mask 15% in Masked Language Modeling?cited by 118
- 2020Multilingual Alignment of Contextual Word Representationscited by 114
- 2020Finding Universal Grammatical Relations in Multilingual BERTcited by 110
- 2020Perturbed Masking: Parameter-free Probing for Analyzing and Interpreting BERTcited by 109
- 2021Transformer visualization via dictionary learning: contextualized embedding as a linear superposition of transformer factorscited by 108
- 2020When Do You Need Billions of Words of Pretraining Data?cited by 104
- 2020Syntactic Structure from Deep Learningcited by 99
- 2020Analyzing Individual Neurons in Pre-trained Language Modelscited by 97
- 2024Information Flow Routes: Automatically Interpreting Language Models at Scalecited by 95
50 most cited of 223 All 223 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.