mlsanityData to 7 Oct 2026

Circuit Breaking: Removing Model Behaviors with Targeted Ablation

arXiv 2309.05973 ↗First posted 12 Sep 2023cs.CL cs.LGcited by 27

Not on the map. Why?

Abstract (arXiv)

Language models often exhibit behaviors that improve performance on a pre-training objective but harm performance on downstream tasks. We propose a novel approach to removing undesirable behaviors by ablating a small number of causal pathways between model components, with the intention of disabling the computational circuit responsible for the bad behavior. Given a small dataset of inputs where the model behaves poorly, we learn to ablate a small number of important causal pathways. In the setting of reducing GPT-2 toxic language generation, we find ablating just 12 of the 11.6K causal edges mitigates toxic generation with minimal degradation of performance on other inputs.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Builds on 35 of its 61 references that are in the map. First posted on arXiv 12 Sep 2023.

Builds on

35 of 61 references.

References · 35

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2023GPT-4 Technical Reportcited by 18,132
  4. 2022Training language models to follow instructions with human feedbackcited by 15,712
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  6. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  7. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  8. 2020Learning to summarize from human feedbackcited by 2,562
  9. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
  10. 2022Editing Models with Task Arithmeticcited by 1,076
  11. 2019Machine Unlearningcited by 1,020
  12. 2023Progress measures for grokking via mechanistic interpretabilitycited by 822
23 more
  1. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 700
  2. 2017Interpretable Explanations of Black Boxes by Meaningful Perturbationcited by 611
  3. 2019Eternal Sunshine of the Spotless Net: Selective Forgetting in Deep Networkscited by 599
  4. 2017Learning Sparse Neural Networks through L0 Regularizationcited by 571
  5. 2023Erasing Concepts from Diffusion Modelscited by 530
  6. 2021Remember What You Want to Forget: Algorithms for Machine Unlearningcited by 360
  7. 2023Finding Neurons in a Haystack: Case Studies with Sparse Probingcited by 328
  8. 2018Manipulating and Measuring Model Interpretabilitycited by 256
  9. 2018Learning to Explain: An Information-Theoretic Perspective on Model Interpretationcited by 248
  10. 2017Real Time Image Saliency for Black Box Classifierscited by 246
  11. 2021Adversarial Neuron Pruning Purifies Backdoored Deep Modelscited by 230
  12. 2020Explaining by Removing: A Unified Framework for Model Explanationcited by 218
  13. 2019Structured Pruning of Large Language Modelscited by 201
  14. 2023Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representationscited by 197
  15. 2023Localizing Model Behavior with Path Patchingcited by 182
  16. 2014Object Detectors Emerge in Deep Scene CNNscited by 178
  17. 2022Toward Transparent AI: A Survey on Interpreting the Inner Structures of Deep Neural Networkscited by 126
  18. 2021Inducing Causal Structure for Interpretable Neural Networkscited by 107
  19. 2020Neuron Shapley: Discovering the Responsible Neuronscited by 98
  20. 2021Low-Complexity Probing via Finding Subnetworkscited by 63
  21. 2022Adversarial Training for High-Stakes Reliabilitycited by 60
  22. 2021Few-shot Backdoor Defense Using Shapley Estimationcited by 40
  23. 2020Raiders of the Lost Kek: 3.5 Years of Augmented 4chan Posts from the Politically Incorrect Boardcited by 24

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 18; 2025: 5; 2026: 2.

27 papers.

Sort:

By area:

  1. 2024Improving Alignment and Robustness with Circuit Breakerscited by 302
  2. 2024Rethinking Machine Unlearning for Large Language Modelscited by 247
  3. 2024Towards Principled Evaluations of Sparse Autoencoders for Interpretability and Controlcited by 83
  4. 2024Knowledge Circuits in Pretrained Transformerscited by 60
  5. 2024How to think step-by-step: A mechanistic understanding of chain-of-thought reasoningcited by 53
  6. 2024pyvene: A Library for Understanding and Improving PyTorch Models via Interventionscited by 53
  7. 2024Knowledge Mechanisms in Large Language Models: A Survey and Perspectivecited by 50
  8. 2023SoK: Memorization in General-Purpose Large Language Modelscited by 36
  9. 2024Mechanistic Unlearning: Robust Knowledge Unlearning and Editing via Mechanistic Localizationcited by 34
  10. 2024The Quest for the Right Mediator: Surveying Mechanistic Interpretability Through the Lens of Causal Mediation Analysiscited by 24
  11. 2024Missed Causes and Ambiguous Effects: Counterfactuals Pose Challenges for Interpreting Neural Networkscited by 21
  12. 2024Optimal ablation for interpretabilitycited by 21
15 more
  1. 2024Decomposing and Editing Predictions by Modeling Model Computationcited by 19
  2. 2024Transformer Circuit Faithfulness Metrics are not Robustcited by 18
  3. 2023Discovering Knowledge-Critical Subnetworks in Pretrained Language Modelscited by 16
  4. 2024HiddenGuard: Fine-Grained Safe Generation with Specialized Representation Routercited by 16
  5. 2024Decoding Dark Matter: Specialized Sparse Autoencoders for Interpreting Rare Concepts in Foundation Modelscited by 15
  6. 2024The Heuristic Core: Understanding Subnetwork Generalization in Pretrained Language Modelscited by 9
  7. 2025IF-GUIDE: Influence Function-Guided Detoxification of LLMscited by 7
  8. 2024When Parts Are Greater Than Sums: Individual LLM Components Can Outperform Full Modelscited by 6
  9. 2025Unboxing the Black Box: Mechanistic Interpretability for Algorithmic Understanding of Neural Networkscited by 6
  10. 2025Interpretability as Alignment: Making Internal Understanding a Design Principlecited by 3
  11. 2025Grounding or Guessing? Visual Signals for Detecting Hallucinations in Sign Language Translationcited by 3
  12. 2024Rule Extrapolation in Language Models: A Study of Compositional Generalization on OOD Promptscited by 2
  13. 2026Compressed Sensing for Capability Localization in Large Language Modelscited by 2
  14. 2025Promote, Suppress, Iterate: How Language Models Answer One-to-Many Factual Queriescited by 1
  15. 2026Who Do Language Models Think Is Competent? A Mechanistic Analysis of Occupational Biascited by 0