mlsanityData to 7 Oct 2026

InterpBench: Semi-Synthetic Transformers for Evaluating Mechanistic Interpretability Techniques

arXiv 2407.14494 ↗First posted 19 Jul 2024cs.LGcited by 24

Not on the map. Why?

Abstract (arXiv)

Mechanistic interpretability methods aim to identify the algorithm a neural network implements, but it is difficult to validate such methods when the true algorithm is unknown. This work presents InterpBench, a collection of semi-synthetic yet realistic transformers with known circuits for evaluating these techniques. We train simple neural networks using a stricter version of Interchange Intervention Training (IIT) which we call Strict IIT (SIIT). Like the original, SIIT trains neural networks by aligning their internal computation with a desired high-level causal model, but it also prevents non-circuit nodes from affecting the model's output. We evaluate SIIT on sparse transformers produced by the Tracr tool and find that SIIT models maintain Tracr's original circuit while being more realistic. SIIT can also train transformers with larger circuits, like Indirect Object Identification (IOI). Finally, we use our benchmark to evaluate existing circuit discovery techniques.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Builds on 32 of its 57 references that are in the map. First posted on arXiv 19 Jul 2024.

Builds on

32 of 57 references.

References · 32

  1. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,359
  2. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,130
  3. 2022In-context Learning and Induction Headscited by 930
  4. 2022Toy Models of Superpositioncited by 928
  5. 2023Progress measures for grokking via mechanistic interpretabilitycited by 822
  6. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 700
  7. 2024Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Modelscited by 412
  8. 2021Causal Abstractions of Neural Networkscited by 401
  9. 2023Towards Best Practices of Activation Patching in Language Models: Metrics and Methodscited by 328
  10. 2024Not All Language Model Features Are One-Dimensionally Linearcited by 201
  11. 2023How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language modelcited by 199
  12. 2023Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representationscited by 197
20 more
  1. 2023Attribution Patching Outperforms Automated Circuit Discoverycited by 196
  2. 2021Thinking Like Transformerscited by 182
  3. 2024Improving Dictionary Learning with Gated Sparse Autoencoderscited by 171
  4. 2023The Clock and the Pizza: Two Stories in Mechanistic Explanation of Neural Networkscited by 153
  5. 2023Does Circuit Analysis Interpretability Scale? Evidence from Multiple Choice Capabilities in Chinchillacited by 145
  6. 2023A Toy Model of Universality: Reverse Engineering How Networks Learn Group Operationscited by 141
  7. 2021Inducing Causal Structure for Interpretable Neural Networkscited by 107
  8. 2023Interpretability at Scale: Identifying Causal Mechanisms in Alpacacited by 102
  9. 2024AtP*: An efficient and scalable method for localizing LLM behaviour to componentscited by 100
  10. 2023Tracr: Compiled Transformers as a Laboratory for Interpretabilitycited by 99
  11. 2024RAVEL: Evaluating Interpretability Methods on Disentangling Language Model Representationscited by 77
  12. 2022Polysemanticity and Capacity in Neural Networkscited by 74
  13. 2024Identifying Functionally Important Features with End-to-End Sparse Dictionary Learningcited by 64
  14. 2021Low-Complexity Probing via Finding Subnetworkscited by 63
  15. 2024A Multimodal Automated Interpretability Agentcited by 62
  16. 2024A Mechanistic Analysis of a Transformer Trained on a Symbolic Multi-Step Reasoning Taskcited by 49
  17. 2024CausalGym: Benchmarking causal interpretability methods on linguistic taskscited by 45
  18. 2023FIND: A Function Description Benchmark for Evaluating Interpretability Methodscited by 32
  19. 2023Look Before You Leap: A Universal Emergent Decomposition of Retrieval Tasks in Language Modelscited by 17
  20. 2024The Local Interaction Basis: Identifying Computationally-Relevant and Sparsely Interacting Features in Neural Networkscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 6; 2026: 18.

24 papers.

Sort:

By area:

  1. 2026The Curse of Multiple Mediators: Hidden Interaction Effects in Activation Patchingcited by 9
  2. 2026All Circuits Lead to Rome: Rethinking Functional Anisotropy in Circuit and Sheaf Discovery for LLMscited by 8
  3. 2026SynthSAEBench: Evaluating Sparse Autoencoders on Scalable Realistic Synthetic Datacited by 7
  4. 2025Circuit Stability Characterizes Language Model Generalizationcited by 6
  5. 2025Large Language Model Sourcing: A Surveycited by 5
  6. 2025Evaluating Explanations: An Explanatory Virtues Framework for Mechanistic Interpretability -- The Strange Science Part I.iicited by 2
  7. 2026Make Mechanistic Interpretability Auditable: A Call to Develop Guidelines via Continuous Collaborative Reviewingcited by 2
  8. 2026When Attribution Patching Lies: Diagnosis and a Second-Order Correctioncited by 2
  9. 2026Scalable Circuit Learning for Interpreting Large Language Modelscited by 2
  10. 2026Circuit Claims Depend on What Is Extracted and How It Is Comparedcited by 2
  11. 2026Can Neural Networks Learn by Experimenting on Themselves? Self-Interventional Learning from Functional Consequences to Predictive Self-Knowledgecited by 2
  12. 2026RUBEN: Rule-Based Explanations for Retrieval-Augmented LLM Systemscited by 1
12 more
  1. 2026Can Language Model Agents be Helpful Circuit Explainers in Mechanistic Interpretability?cited by 1
  2. 2026Resist and Update: Counterfactual Report Coordinates for Incentive-Compatible LLMscited by 1
  3. 2026ObserverBench: Testing Mechanistic Estimates for Intervention and Controlcited by 1
  4. 2025`A la recherche du sens perdu: your favourite LLM might have more to say than you can understandcited by 0
  5. 2025Inducing Causal Structure for Interpretable Neural Networks Applied to Glucose Prediction for T1DM Patientscited by 0
  6. 2025Rule-Based Explanations for Retrieval-Augmented LLM Systemscited by 0
  7. 2026AIMO Interpretability Challengecited by 0
  8. 2026CircuitKIT : Circuit Discovery, Evaluation, and Application Toolkit for Mechanistic Interpretabilitycited by 0
  9. 2026Can Graph Learning Learn Circuits?cited by 0
  10. 2026MAxBench: A Multinomial Concept Recovery Benchmarkcited by 0
  11. 2026Right Answer, Wrong Mechanism: Detecting Pernicious Divergence in Causal Interventionscited by 0
  12. 2026Are We Recovering Mechanisms? Objective-Level Recovery Gaps in Mechanistic Interpretabilitycited by 0