mlsanityData to 14 Oct 2026

Circuit-Diff: Factual Edit-based Intervention Method for Localizing Knowledge in Attribution Graphs

arXiv 2609.23892 ↗First posted 20 Sep 2026cs.LG cs.AIcited by 0

On the map: Mechanistic interpretability → Circuits and mechanistic analysis

Abstract (arXiv)

Mechanistic interpretability defines features as the fundamental units of a neural network and circuits as the weighted subgraphs that carry out its computation. Because individual neurons are polysemantic, Cross-Layer Transcoders (CLTs) were introduced as a way to approximate a model's circuits by generating an attribution graph. The nodes of that graph, however, are unlabeled features: reading a graph means pruning it and then working out by hand what each surviving node means. To make CLTs easier to use for circuit discovery, we introduce Circuit-Diff, which intervenes on the model itself with a low-rank factual edit and takes the features whose role in the attribution graph changes under that edit as related to the edited knowledge. On the edits we examine, the flagged nodes are not only detectors of the object token: read off the CLT's released feature dashboards, they include features for the history, geography and associations surrounding the old and new objects. We formalize the method, measure how reliable a frozen CLT remains after a factual edit, test the selected nodes causally by patching them on up to 24 CounterFact edits, give a case study, and release an open-source implementation built on the circuit-tracer package, together with two further tools (multi-prompt aggregation and rule-based supernode labeling).

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Circuits and mechanistic analysis, one of 13 directions in Mechanistic interpretability. Builds on 19 of its 30 references that are in the map. First posted on arXiv 20 Sep 2026.

Builds on

19 of 30 references.

Closest ancestors · 2

Same direction, newest first

  1. 2026Prune, Interpret, Evaluate: A Cross-Layer Transcoder-Native Framework for Efficient Circuit Discovery via Feature Attribution18 Apr
  2. 2024Transcoders Find Interpretable LLM Feature Circuits17 Jun

Other references · 17

  1. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  2. 2022Mass-Editing Memory in a Transformercited by 1,001
  3. 2022Toy Models of Superpositioncited by 971
  4. 2023Dissecting Recall of Factual Associations in Auto-Regressive Language Modelscited by 528
  5. 2023Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Modelscited by 243
  6. 2016Linear Algebraic Structure of Word Senses, with Applications to Polysemycited by 165
  7. 2024Automatically Interpreting Millions of Features in Large Language Modelscited by 113
  8. 2022Understanding Transformer Memorization Recall Through Idiomscited by 62
  9. 2023Detecting Edit Failures In Large Language Models: An Improved Specificity Benchmarkcited by 62
  10. 2024Understanding Factual Recall in Transformers via Associative Memoriescited by 51
  11. 2023Rigorously Assessing Natural Language Explanations of Neuronscited by 45
  12. 2024Long-form evaluation of model editingcited by 23
5 more
  1. 2026Circuit Tracing in Vision-Language Models: Understanding the Internal Mechanisms of Multimodal Thinkingcited by 8
  2. 2024Understanding the Collapse of LLMs in Model Editingcited by 3
  3. 2026DifFRACT: Diffusion Feature Reconstruction and Attribution for Circuit Tracingcited by 3
  4. 2025Automated Attribution Graph Interpretation via Probe Promptingcited by 2
  5. 2026CRaFT: Circuit-Guided Refusal Feature Selection via Cross-Layer Transcoderscited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023Attribution Patching Outperforms Automated Circuit Discovery
  2. 2026ADAG: Automatically Describing Attribution Graphs
  3. 2024Knowledge Circuits in Pretrained Transformers
  4. 2025Attribution-Guided Pruning for Insight and Control: Circuit Discovery and Targeted Correction in Small-scale LLMs
  5. 2025Circuit Insights: Towards Interpretability Beyond Activations
  6. 2026CLT-Forge: A Scalable Library for Cross-Layer Transcoders and Attribution Graphs
  7. 2024Understanding Language Model Circuits through Knowledge Editing
  8. 2025Discovering Transformer Circuits via a Hybrid Attribution and Pruning Framework