mlsanityData to 14 Oct 2026

Attribution-Guided Pruning for Insight and Control: Circuit Discovery and Targeted Correction in Small-scale LLMs

arXiv 2506.13727 ↗First posted 16 Jun 2025cs.LG cs.AI cs.CLcited by 3

On the map: Mechanistic interpretability → Circuits and mechanistic analysis

Abstract (arXiv)

Large Language Models (LLMs) are widely deployed in real-world applications, yet their internal mechanisms remain difficult to interpret and control, limiting our ability to diagnose and correct undesirable behaviors. Mechanistic interpretability addresses this challenge by identifying circuits -- subsets of model components responsible for specific behaviors. However, discovering such circuits in LLMs remains difficult due to their scale and complexity. We frame circuit discovery as identifying parameters that contribute most to model outputs on task-specific inputs, and use Layer-wise Relevance Propagation (LRP) with reference samples to attribute and extract these components via pruning. Building on this, we introduce contrastive relevance to isolate circuits associated with undesired behaviors while preserving general capabilities, enabling targeted model correction. On OPT-125M, we show that pruning as little as ~0.3% of neurons substantially reduces toxic outputs, while pruning approximately 0.03% of weight elements mitigates repetitive text generation without degrading general performance. These results establish attribution-guided pruning as an effective mechanism for identifying and intervening on behavior-specific circuits in LLMs. We further validate our findings on additional small-scale language models, demonstrating that the proposed approach transfers across architectures. Our code is publicly available at https://github.com/erfanhatefi/SparC3.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Circuits and mechanistic analysis, one of 13 directions in Mechanistic interpretability. Builds on 37 of its 46 references that are in the map. First posted on arXiv 16 Jun 2025.

Builds on

37 of 46 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023Attribution Patching Outperforms Automated Circuit Discovery16 Oct
  2. 2023Towards Automated Circuit Discovery for Mechanistic Interpretability28 Apr

Other references · 35

  1. 2017Attention Is All You Needcited by 60,361
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  5. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  6. 2016Pointer Sentinel Mixture Modelscited by 3,096
  7. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  8. 2017Learning Important Features Through Propagating Activation Differencescited by 1,458
  9. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
  10. 2017SmoothGrad: removing noise by adding noisecited by 1,109
  11. 2022Mass-Editing Memory in a Transformercited by 1,001
  12. 2023Steering Language Models With Activation Engineeringcited by 922
23 more
  1. 2023A Simple and Effective Pruning Approach for Large Language Modelscited by 800
  2. 2024TinyLlama: An Open-Source Small Language Modelcited by 618
  3. 2021Knowledge Neurons in Pretrained Transformerscited by 601
  4. 2020Null It Out: Guarding Protected Attributes by Iterative Nullspace Projectioncited by 455
  5. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 435
  6. 2015Evaluating the visualization of what a Deep Neural Network has learnedcited by 419
  7. 2023Trustworthy LLMs: a Survey and Guideline for Evaluating Large Language Models' Alignmentcited by 401
  8. 2017Right for the Right Reasons: Training Differentiable Models by Constraining their Explanationscited by 294
  9. 2019Structured Pruning of Large Language Modelscited by 203
  10. 2017The Shattered Gradients Problem: If resnets are the answer, then what is the question?cited by 157
  11. 2020Making deep neural networks right for the right scientific reasons by interacting with their explanationscited by 142
  12. 2024AttnLRP: Attention-Aware Layer-Wise Relevance Propagation for Transformerscited by 120
  13. 2022XAI for Transformers: Better Explanations through Conservative Propagationcited by 100
  14. 2024Information Flow Routes: Automatically Interpreting Language Models at Scalecited by 95
  15. 2019Pruning by Explaining: A Novel Criterion for Deep Neural Network Pruningcited by 89
  16. 2024LLM Circuit Analyses Are Consistent Across Training and Scalecited by 63
  17. 2021Software for Dataset-wide XAI: From Local Explanations to Global Insights with Zennit, CoRelAy, and ViRelAycited by 55
  18. 2024MambaLRP: Explaining Selective State Space Sequence Modelscited by 32
  19. 2023From Hope to Safety: Unlearning Biases of Deep Models via Gradient Penalization in Latent Spacecited by 20
  20. 2023Reveal to Revise: An Explainable AI Life Cycle for Iterative Bias Correction of Deep Modelscited by 17
  21. 2021ECQx: Explainability-Driven Quantization for Low-Bit and Sparse DNNscited by 15
  22. 2024Pruning By Explaining Revisited: Optimizing Attribution Methods to Prune CNNs and Transformerscited by 11
  23. 2025A Close Look at Decomposition-based XAI-Methods for Transformer Language Modelscited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 2; 2026: 1.

3 papers.

  1. 2025RelP: Faithful and Efficient Circuit Discovery in Language Models via Relevance Patchingcited by 13
  2. 2025Interpretability as Alignment: Making Internal Understanding a Design Principlecited by 3
  3. 2026ReCal3R: Reliability-Calibrated Learning Rates for Streaming 3D Reconstructioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Multi-Granular Node Pruning for Causal Circuit Discovery
  2. 2025Towards Understanding Fine-Tuning Mechanisms of LLMs via Circuit Analysis
  3. 2026Exploring the Limits of Pruning: Task-Specific Neurons, Model Collapse, and Recovery in Task-Specific Large Language Models
  4. 2024Dissecting Language Models: Machine Unlearning via Selective Pruning
  5. 2024Extracting Interpretable Task-Specific Circuits from Large Language Models for Faster Inference
  6. 2026Causally Grounded Mechanistic Interpretability for LLMs with Faithful Natural-Language Explanations
  7. 2026RESCUE: Repairing Language Model Errors to Sparse Circuits via Reinforcement Learning
  8. 2026Mechanistic Interpretability for Large Language Model Alignment: Progress, Challenges, and Future Directions