mlsanityData to 14 Oct 2026

Causal Interventions Reveal Typologically Organized Syntactic Mechanisms in Multilingual Language Models

arXiv 2608.28924 ↗First posted 28 Aug 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Linguistic theory has long recognized cross-linguistic syntactic regularities, leading to claims that these similar structures are processed by similar mechanisms. However, this hypothesis has been difficult to test empirically due to our lack of fine-grained, manipulable access of human processing mechanisms. In this work, we take advantage of techniques from mechanistic interpretability to study such a question in multilingual LMs. We first isolate language-internal mechanisms before attempting to transfer them cross-lingually. Across four models and three well-studied constructions (subject--verb number agreement, anaphoric pronoun gender agreement, and filler--gap object extraction) we find consistent cross-lingual mechanism transfer. We further find transfer to be graded, with more transfer between more typologically similar languages. We believe our work provides novel hypotheses about cross-linguistic syntactic structures and multilingual processing, and more broadly shows how the study of language models can help inform linguistic theory.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 34 of its 59 references that are in the map. First posted on arXiv 28 Aug 2026.

Builds on

34 of 59 references.

References · 34

  1. 2024The Llama 3 Herd of Modelscited by 14,715
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2019How multilingual is Multilingual BERT?cited by 645
  4. 2019Designing and Interpreting Probes with Control Taskscited by 640
  5. 2019BLiMP: The Benchmark of Linguistic Minimal Pairs for Englishcited by 564
  6. 2016Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependenciescited by 354
  7. 2020How Good is Your Tokenizer? On the Monolingual Performance of Multilingual Language Modelscited by 318
  8. 2020Information-Theoretic Probing with Minimum Description Lengthcited by 232
  9. 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretabilitycited by 205
  10. 2020A Systematic Assessment of Syntactic Generalization in Neural Language Modelscited by 165
  11. 2022mGPT: Few-Shot Learners Go Multilingualcited by 142
  12. 2020On the Predictive Power of Neural Language Models for Human Real-Time Comprehension Behaviorcited by 119
22 more
  1. 2022Why Does Surprisal From Larger Transformer-Based Language Models Provide a Poorer Fit to Human Reading Times?cited by 113
  2. 2020Finding Universal Grammatical Relations in Multilingual BERTcited by 110
  3. 2020Syntactic Structure from Deep Learningcited by 99
  4. 2022What Artificial Neural Networks Can Tell Us About Human Language Acquisitioncited by 99
  5. 2025Sparse Autoencoders Trained on the Same Data Learn Different Featurescited by 93
  6. 2025Sparse Autoencoders Do Not Find Canonical Units of Analysiscited by 88
  7. 2018What do RNN Language Models Learn about Filler-Gap Dependencies?cited by 86
  8. 2025MultiBLiMP 1.0: A Massively Multilingual Benchmark of Linguistic Minimal Pairscited by 71
  9. 2023Tokenization Impacts Multilingual Language Modeling: Assessing Vocabulary Allocation and Overlap Across Languagescited by 62
  10. 2022minicons: Enabling Flexible Behavioral and Representational Analyses of Transformer Language Modelscited by 58
  11. 2021Lower Perplexity is Not Always Human-Likecited by 52
  12. 2025How Linguistics Learned to Stop Worrying and Love the Language Modelscited by 51
  13. 2024CausalGym: Benchmarking causal interpretability methods on linguistic taskscited by 46
  14. 2021Deep Subjecthood: Higher-Order Grammatical Features in Multilingual BERTcited by 41
  15. 2023Transformer-Based Language Model Surprisal Predicts Human Reading Times Best with About Two Billion Training Tokenscited by 41
  16. 2024Frequency Explains the Inverse Correlation of Large Language Models' Size, Training Data Amount, and Surprisal's Fit to Reading Timescited by 39
  17. 2020It's not Greek to mBERT: Inducing Word-Level Translations from Multilingual BERTcited by 30
  18. 2025Large Language Models Are Human-Like Internallycited by 29
  19. 2024On the Similarity of Circuits across Languages: a Case Study on the Subject-verb Agreement Taskcited by 24
  20. 2025Causal Interventions Reveal Shared Structure Across English Filler-Gap Constructionscited by 21
  21. 2024A systematic framework for generating novel experimental hypotheses from language modelscited by 9
  22. 2025Tracing Multilingual Representations in LLMs with Cross-Layer Transcoderscited by 8

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024The Same But Different: Structural Similarities and Differences in Multilingual Language Modeling
  2. 2026Fine-Grained Analysis of Shared Syntactic Mechanisms in Language Models
  3. 2026Shared Circuits for Shared Grammar: Tracing Subject-Verb Agreement Across Languages
  4. 2026Multilingual Language Models Encode Script Over Linguistic Structure
  5. 2025Large Language Models Share Representations of Latent Grammatical Concepts Across Typologically Diverse Languages
  6. 2026Lingua Franca or Probing Artifact? Rethinking Latent Language in Multilingual LLMs
  7. 2025Are Knowledge and Reference in Multilingual Language Models Cross-Lingually Consistent?
  8. 2025Lost in Multilinguality: Dissecting Cross-lingual Factual Inconsistency in Transformer Language Models