# Circuit Compositions: Exploring Modular Structures in Transformer-Based Language Models

arXiv: https://arxiv.org/abs/2410.01434 · HTML: https://mlsanity.com/paper/2410.01434/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 50 references that are in the map.
- First posted on arXiv 2 Oct 2024.

## Builds on (most cited first, 30 of 34)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Locating and Editing Factual Associations in GPT](https://mlsanity.com/paper/2202.05262/) (2022)
- [GLU Variants Improve Transformer](https://mlsanity.com/paper/2002.05202/) (2020)
- [Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small](https://mlsanity.com/paper/2211.00593/) (2022)
- [Toy Models of Superposition](https://mlsanity.com/paper/2209.10652/) (2022)
- [Progress measures for grokking via mechanistic interpretability](https://mlsanity.com/paper/2301.05217/) (2023)
- [AdapterFusion: Non-Destructive Task Composition for Transfer Learning](https://mlsanity.com/paper/2005.00247/) (2020)
- [Towards Automated Circuit Discovery for Mechanistic Interpretability](https://mlsanity.com/paper/2304.14997/) (2023)
- [Learning Sparse Neural Networks through L0 Regularization](https://mlsanity.com/paper/1712.01312/) (2017)
- [Dissecting Recall of Factual Associations in Auto-Regressive Language Models](https://mlsanity.com/paper/2304.14767/) (2023)
- [The State of Sparsity in Deep Neural Networks](https://mlsanity.com/paper/1902.09574/) (2019)
- [Towards Best Practices of Activation Patching in Language Models: Metrics and Methods](https://mlsanity.com/paper/2309.16042/) (2023)
- [Direct and Indirect Effects](https://mlsanity.com/paper/1301.2300/) (2013)
- [Compositionality decomposed: how do neural networks generalise?](https://mlsanity.com/paper/1908.08351/) (2019)
- [AdapterDrop: On the Efficiency of Adapters in Transformers](https://mlsanity.com/paper/2010.11918/) (2020)
- [Attribution Patching Outperforms Automated Circuit Discovery](https://mlsanity.com/paper/2310.10348/) (2023)
- [How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model](https://mlsanity.com/paper/2305.00586/) (2023)
- [Thinking Like Transformers](https://mlsanity.com/paper/2106.06981/) (2021)
- [Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Tracking](https://mlsanity.com/paper/2402.14811/) (2024)
- [Have Faith in Faithfulness: Going Beyond Circuit Overlap When Finding Model Mechanisms](https://mlsanity.com/paper/2403.17806/) (2024)
- [Composable Sparse Fine-Tuning for Cross-Lingual Transfer](https://mlsanity.com/paper/2110.07560/) (2021)
- [Circuit Component Reuse Across Tasks in Transformer Language Models](https://mlsanity.com/paper/2310.08744/) (2023)
- [Tracr: Compiled Transformers as a Laboratory for Interpretability](https://mlsanity.com/paper/2301.05062/) (2023)
- [Winning the Lottery with Continuous Sparsification](https://mlsanity.com/paper/1912.04427/) (2019)
- [Are Neural Nets Modular? Inspecting Functional Modularity Through Differentiable Weight Masks](https://mlsanity.com/paper/2010.02066/) (2020)
- [A Primer on the Inner Workings of Transformer-based Language Models](https://mlsanity.com/paper/2405.00208/) (2024)
- [Training Sparse Neural Networks](https://mlsanity.com/paper/1611.06694/) (2016)
- [Low-Complexity Probing via Finding Subnetworks](https://mlsanity.com/paper/2104.03514/) (2021)
- [Finding Transformer Circuits with Edge Pruning](https://mlsanity.com/paper/2406.16778/) (2024)

## Built on by (most cited first, 11 of 11)

- [Same Task, Different Circuits: Disentangling Modality-Specific Mechanisms in VLMs](https://mlsanity.com/paper/2506.09047/) (2025)
- [Discovering Transformer Circuits via a Hybrid Attribution and Pruning Framework](https://mlsanity.com/paper/2510.03282/) (2025)
- [Differentiable Faithfulness Alignment for Cross-Model Circuit Transfer](https://mlsanity.com/paper/2604.24302/) (2026)
- [Not Just RLHF: Why Alignment Alone Won't Fix Multi-Agent Sycophancy](https://mlsanity.com/paper/2605.12991/) (2026)
- [What Happens Inside Agent Memory? Circuit Analysis from Emergence to Diagnosis](https://mlsanity.com/paper/2605.03354/) (2026)
- [How Much Do Circuits Tell Us? Measuring the Consistency and Specificity of Language Model Circuits](https://mlsanity.com/paper/2605.08348/) (2026)
- [Circuit Claims Depend on What Is Extracted and How It Is Compared](https://mlsanity.com/paper/2607.18921/) (2026)
- [Seeing Through Circuits: Faithful Mechanistic Interpretability for Vision Transformers](https://mlsanity.com/paper/2604.14477/) (2026)
- [Inside-Out: Measuring Generalization in Vision Transformers Through Inner Workings](https://mlsanity.com/paper/2604.08192/) (2026)
- [CORTEX: Learning to Share and Specialize in Dense Language Models](https://mlsanity.com/paper/2609.34449/) (2026)
- [CircuitTree: Composing Circuits into A Syllable Transcriber](https://mlsanity.com/paper/2610.12520/) (2026)

## Abstract (arXiv)

A fundamental question in interpretability research is to what extent neural networks, particularly language models, implement reusable functions through subnetworks that can be composed to perform more complex tasks. Recent advances in mechanistic interpretability have made progress in identifying textitcircuits, which represent the minimal computational subgraphs responsible for a model's behavior on specific tasks. However, most studies focus on identifying circuits for individual tasks without investigating how functionally similar circuits textitrelate to each other. To address this gap, we study the modularity of neural networks by analyzing circuits for highly compositional subtasks within a transformer-based language model. Specifically, given a probabilistic context-free grammar, we identify and compare circuits responsible for ten modular string-edit operations. Our results indicate that functionally similar circuits exhibit both notable node overlap and cross-task faithfulness. Moreover, we demonstrate that the circuits identified can be reused and combined through set operations to represent more complex functional model capabilities.