# Finding Transformer Circuits with Edge Pruning

arXiv: https://arxiv.org/abs/2406.16778 · HTML: https://mlsanity.com/paper/2406.16778/

## Key facts

- Cited by 60 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 44 references that are in the map.
- First posted on arXiv 24 Jun 2024.

## Builds on (most cited first, 30 of 32)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023)
- [Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small](https://mlsanity.com/paper/2211.00593/) (2022)
- [Are Sixteen Heads Really Better than One?](https://mlsanity.com/paper/1905.10650/) (2019)
- [Towards Automated Circuit Discovery for Mechanistic Interpretability](https://mlsanity.com/paper/2304.14997/) (2023)
- [Channel Pruning for Accelerating Very Deep Neural Networks](https://mlsanity.com/paper/1707.06168/) (2017)
- [Learning Sparse Neural Networks through L0 Regularization](https://mlsanity.com/paper/1712.01312/) (2017)
- [Sheared LLaMA: Accelerating Language Model Pre-training via Structured Pruning](https://mlsanity.com/paper/2310.06694/) (2023)
- [Towards Best Practices of Activation Patching in Language Models: Metrics and Methods](https://mlsanity.com/paper/2309.16042/) (2023)
- [Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representations](https://mlsanity.com/paper/2303.02536/) (2023)
- [Attribution Patching Outperforms Automated Circuit Discovery](https://mlsanity.com/paper/2310.10348/) (2023)
- [Structured Pruning of Large Language Models](https://mlsanity.com/paper/1910.04732/) (2019)
- [How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model](https://mlsanity.com/paper/2305.00586/) (2023)
- [Localizing Model Behavior with Path Patching](https://mlsanity.com/paper/2304.05969/) (2023)
- [Thinking Like Transformers](https://mlsanity.com/paper/2106.06981/) (2021)
- [Structured Pruning Learns Compact and Accurate Models](https://mlsanity.com/paper/2204.00408/) (2022)
- [The Hydra Effect: Emergent Self-repair in Language Model Computations](https://mlsanity.com/paper/2307.15771/) (2023)
- [Does Circuit Analysis Interpretability Scale? Evidence from Multiple Choice Capabilities in Chinchilla](https://mlsanity.com/paper/2307.09458/) (2023)
- [Fine-Tuning Enhances Existing Mechanisms: A Case Study on Entity Tracking](https://mlsanity.com/paper/2402.14811/) (2024)
- [Have Faith in Faithfulness: Going Beyond Circuit Overlap When Finding Model Mechanisms](https://mlsanity.com/paper/2403.17806/) (2024)
- [Mechanistically analyzing the effects of fine-tuning on procedurally defined tasks](https://mlsanity.com/paper/2311.12786/) (2023)
- [AtP*: An efficient and scalable method for localizing LLM behaviour to components](https://mlsanity.com/paper/2403.00745/) (2024)
- [Tracr: Compiled Transformers as a Laboratory for Interpretability](https://mlsanity.com/paper/2301.05062/) (2023)
- [Interpretability at Scale: Identifying Causal Mechanisms in Alpaca](https://mlsanity.com/paper/2305.08809/) (2023)
- [Is This the Subspace You Are Looking for? An Interpretability Illusion for Subspace Activation Patching](https://mlsanity.com/paper/2311.17030/) (2023)
- [Low-Complexity Probing via Finding Subnetworks](https://mlsanity.com/paper/2104.03514/) (2021)
- [Learning Transformer Programs](https://mlsanity.com/paper/2306.01128/) (2023)
- [Pruning-as-Search: Efficient Neural Architecture Search via Channel Pruning and Structural Reparameterization](https://mlsanity.com/paper/2206.01198/) (2022)
- [VISIT: Visualizing and Interpreting the Semantic Information Flow of Transformers](https://mlsanity.com/paper/2305.13417/) (2023)

## Built on by (most cited first, 30 of 60)

- [A Practical Review of Mechanistic Interpretability for Transformer-Based Language Models](https://mlsanity.com/paper/2407.02646/) (2024)
- [Weight-sparse transformers have interpretable circuits](https://mlsanity.com/paper/2511.13653/) (2025)
- [Activation Scaling for Steering and Interpreting Language Models](https://mlsanity.com/paper/2410.04962/) (2024)
- [Optimal ablation for interpretability](https://mlsanity.com/paper/2409.09951/) (2024)
- [Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis](https://mlsanity.com/paper/2502.11812/) (2025)
- [Geometry of Lightning Self-Attention: Identifiability and Dimension](https://mlsanity.com/paper/2408.17221/) (2024)
- [Mechanistic Interpretability as Statistical Estimation: A Variance Analysis](https://mlsanity.com/paper/2510.00845/) (2025)
- [Sheaf Discovery with Joint Computation Graph Pruning and Flexible Granularity](https://mlsanity.com/paper/2407.03779/) (2024)
- [Circuit Compositions: Exploring Modular Structures in Transformer-Based Language Models](https://mlsanity.com/paper/2410.01434/) (2024)
- [Are formal and functional linguistic mechanisms dissociated in language models?](https://mlsanity.com/paper/2503.11302/) (2025)
- [Formal Mechanistic Interpretability: Automated Circuit Discovery with Provable Guarantees](https://mlsanity.com/paper/2602.16823/) (2026)
- [EfficientLLM: Scalable Pruning-Aware Pretraining for Architecture-Agnostic Edge Language Models](https://mlsanity.com/paper/2502.06663/) (2025)
- [The Validation Gap: A Mechanistic Analysis of How Language Models Compute Arithmetic but Fail to Validate It](https://mlsanity.com/paper/2502.11771/) (2025)
- [Rethinking Circuit Completeness in Language Models: AND, OR, and ADDER Gates](https://mlsanity.com/paper/2505.10039/) (2025)
- [Interpretation Meets Safety: A Survey on Interpretation Methods and Tools for Improving LLM Safety](https://mlsanity.com/paper/2506.05451/) (2025)
- [All Circuits Lead to Rome: Rethinking Functional Anisotropy in Circuit and Sheaf Discovery for LLMs](https://mlsanity.com/paper/2605.12671/) (2026)
- [Conditional Co-Ablation: Recovering Self-Repair Backups in Transformer Circuits](https://mlsanity.com/paper/2607.01940/) (2026)
- [An explainable transformer circuit for compositional generalization](https://mlsanity.com/paper/2502.15801/) (2025)
- [CLUE: Conflict-guided Localization for LLM Unlearning Framework](https://mlsanity.com/paper/2509.20977/) (2025)
- [Beyond Components: Singular Vector-Based Interpretability of Transformer Circuits](https://mlsanity.com/paper/2511.20273/) (2025)
- [Llama See, Llama Do: A Mechanistic Perspective on Contextual Entrainment and Distraction in LLMs](https://mlsanity.com/paper/2505.09338/) (2025)
- [Circuit Stability Characterizes Language Model Generalization](https://mlsanity.com/paper/2505.24731/) (2025)
- [Sparse Feature Coactivation Reveals Causal Semantic Modules in Large Language Models](https://mlsanity.com/paper/2506.18141/) (2025)
- [Scaling sparse feature circuit finding for in-context learning](https://mlsanity.com/paper/2504.13756/) (2025)
- [Discovering Transformer Circuits via a Hybrid Attribution and Pruning Framework](https://mlsanity.com/paper/2510.03282/) (2025)
- [Deconstructing Pre-training: Knowledge Attribution Analysis in MoE and Dense Models](https://mlsanity.com/paper/2601.08383/) (2026)
- [SafeSeek: Universal Attribution of Safety Circuits in Language Models](https://mlsanity.com/paper/2603.23268/) (2026)
- [Extracting Interpretable Task-Specific Circuits from Large Language Models for Faster Inference](https://mlsanity.com/paper/2412.15750/) (2024)
- [PAHQ: Accelerating Automated Circuit Discovery through Mixed-Precision Inference Optimization](https://mlsanity.com/paper/2510.23264/) (2025)
- [BlackboxNLP-2025 MIB Shared Task: Exploring Ensemble Strategies for Circuit Localization Methods](https://mlsanity.com/paper/2510.06811/) (2025)

## Abstract (arXiv)

The path to interpreting a language model often proceeds via analysis of circuits -- sparse computational subgraphs of the model that capture specific aspects of its behavior. Recent work has automated the task of discovering circuits. Yet, these methods have practical limitations, as they rely either on inefficient search algorithms or inaccurate approximations. In this paper, we frame automated circuit discovery as an optimization problem and propose *Edge Pruning* as an effective and scalable solution. Edge Pruning leverages gradient-based pruning techniques, but instead of removing neurons or components, it prunes the edges between components. Our method finds circuits in GPT-2 that use less than half the number of edges compared to circuits found by previous methods while being equally faithful to the full model predictions on standard circuit-finding tasks. Edge Pruning is efficient even with as many as 100K examples, outperforming previous methods in speed and producing substantially better circuits. It also perfectly recovers the ground-truth circuits in two models compiled with Tracr. Thanks to its efficiency, we scale Edge Pruning to CodeLlama-13B, a model over 100x the scale that prior methods operate on. We use this setting for a case study comparing the mechanisms behind instruction prompting and in-context learning. We find two circuits with more than 99.96% sparsity that match the performance of the full model and reveal that the mechanisms in the two settings overlap substantially. Our case study shows that Edge Pruning is a practical and scalable tool for interpretability and sheds light on behaviors that only emerge in large models.