mlsanityData to 14 Oct 2026

Identifying a Circuit for Verb Conjugation in GPT-2

arXiv 2506.22105 ↗First posted 27 Jun 2025cs.CL cs.LGcited by 1

Not on the map. Why?

Abstract (arXiv)

I implement a procedure to isolate and interpret the sub-network (or "circuit") responsible for subject-verb agreement in GPT-2 Small. In this study, the model is given prompts where the subject is either singular (e.g. "Alice") or plural (e.g. "Alice and Bob"), and the task is to correctly predict the appropriate verb form ("walks" for singular subjects, "walk" for plural subjects). Using a series of techniques-including performance verification automatic circuit discovery via direct path patching, and direct logit attribution- I isolate a candidate circuit that contributes significantly to the model's correct verb conjugation. The results suggest that only a small fraction of the network's component-token pairs is needed to achieve near-model performance on the base task but substantially more for more complex settings.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 16 of its 26 references that are in the map. First posted on arXiv 27 Jun 2025.

Builds on

16 of 26 references.

References · 16

  1. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  2. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
  3. 2022In-context Learning and Induction Headscited by 964
  4. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 724
  5. 2019BLiMP: The Benchmark of Linguistic Minimal Pairs for Englishcited by 564
  6. 2022Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Spacecited by 532
  7. 2016Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependenciescited by 354
  8. 2019Negated and Misprimed Probes for Pretrained Language Models: Birds Can Talk, But Cannot Flycited by 243
  9. 2023Causal Abstraction: A Theoretical Foundation for Mechanistic Interpretabilitycited by 205
  10. 2023How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language modelcited by 202
  11. 2023Localizing Model Behavior with Path Patchingcited by 195
  12. 2019Assessing BERT's Syntactic Abilitiescited by 192
4 more
  1. 2023Language models are not naysayers: An analysis of language models on negation benchmarkscited by 74
  2. 2024Knowledge Circuits in Pretrained Transformerscited by 60
  3. 2023Red Teaming Deep Neural Networks with Feature Synthesis Toolscited by 21
  4. 2024How does GPT-2 Predict Acronyms? Extracting and Understanding a Circuit via Mechanistic Interpretabilitycited by 14

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026MechRL: Reinforcement Learning Agents Perform Circuit Discovery for Mechanistic Interpretabilitycited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2025Discursive Circuits: How Do Language Models Understand Discourse Relations?
  2. 2024Causal Interventions on Causal Paths: Mapping GPT-2's Reasoning From Syntax to Semantics
  3. 2025Circuit Stability Characterizes Language Model Generalization
  4. 2026Attention as a Routing Graph: Live Circuit Extraction from a Single Forward Pass
  5. 2022Implicit causality in GPT-2: a case study
  6. 2026Interpreting Negation in GPT-2: Layer- and Head-Level Causal Analysis
  7. 2024Adaptive Circuit Behavior and Generalization in Mechanistic Interpretability
  8. 2023A Comprehensive Capability Analysis of GPT-3 and GPT-3.5 Series Models