# Shared Circuits for Shared Grammar: Tracing Subject-Verb Agreement Across Languages

arXiv: https://arxiv.org/abs/2608.18545 · HTML: https://mlsanity.com/paper/2608.18545/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 20 references that are in the map.
- First posted on arXiv 19 Aug 2026.

## Builds on (most cited first, 15 of 15)

- [Towards Best Practices of Activation Patching in Language Models: Metrics and Methods](https://mlsanity.com/paper/2309.16042/) (2023)
- [Function Vectors in Large Language Models](https://mlsanity.com/paper/2310.15213/) (2023)
- [Do Llamas Work in English? On the Latent Language of Multilingual Transformers](https://mlsanity.com/paper/2402.10588/) (2024)
- [Visualizing and Understanding Neural Models in NLP](https://mlsanity.com/paper/1506.01066/) (2015)
- [Language-Specific Neurons: The Key to Multilingual Capabilities in Large Language Models](https://mlsanity.com/paper/2402.16438/) (2024)
- [Separating Tongue from Thought: Activation Patching Reveals Language-Agnostic Concept Representations in Transformers](https://mlsanity.com/paper/2411.08745/) (2024)
- [Refusal Direction is Universal Across Safety-Aligned Languages](https://mlsanity.com/paper/2505.17306/) (2025)
- [The Same But Different: Structural Similarities and Differences in Multilingual Language Modeling](https://mlsanity.com/paper/2410.09223/) (2024)
- [On the Similarity of Circuits across Languages: a Case Study on the Subject-verb Agreement Task](https://mlsanity.com/paper/2410.06496/) (2024)
- [Same Neurons, Different Languages: Probing Morphosyntax in Multilingual Pre-trained Models](https://mlsanity.com/paper/2205.02023/) (2022)
- [Causal Analysis of Syntactic Agreement Neurons in Multilingual Language Models](https://mlsanity.com/paper/2210.14328/) (2022)
- [LinguaLens: Towards Interpreting Linguistic Mechanisms of Large Language Models via Sparse Auto-Encoder](https://mlsanity.com/paper/2502.20344/) (2025)
- [Unveiling Language-Specific Features in Large Language Models via Sparse Autoencoders](https://mlsanity.com/paper/2505.05111/) (2025)
- [UniMorph 4.0: Universal Morphology](https://mlsanity.com/paper/2205.03608/) (2022)
- [Morph Call: Probing Morphosyntactic Content of Multilingual Transformers](https://mlsanity.com/paper/2104.12847/) (2021)

## Abstract (arXiv)

Multilingual large language models often generalize across languages, and prior work suggests that their internal mechanisms can overlap cross-lingually. It remains unclear, however, when such sharing emerges and whether it varies with the overt realization of the same grammatical operation. We investigate this question for present-tense subject-verb agreement, a morphosyntactic process that varies substantially across languages and is only weakly expressed in English. Using activation patching and attention analysis across 29 languages and five open-source model families, we identify the attention heads causally implicated in agreement and compare these head-level signatures across languages. We find that languages with overt person/number inflection exhibit more similar agreement circuitry than non-conjugating languages, with the strongest sharing appearing when the analysis isolates recovery of the inflectional contrast itself. English provides an informative bridge case, becoming more similar to conjugating languages precisely in contexts where overt agreement is required. Finally, many implicated heads display similar attention patterns across languages, suggesting that cross-lingual overlap reflects shared functional roles as well as shared localization. Together, these results indicate that multilingual LLMs reuse partially shared computational structure for morphosyntactic agreement rather than relying on fully separate language-specific solutions.