mlsanityData to 14 Oct 2026

Causally Grounded Mechanistic Interpretability for LLMs with Faithful Natural-Language Explanations

arXiv 2603.09988 ↗First posted 13 Feb 2026cs.CL cs.AIcited by 0

On the map: Mechanistic interpretability → Knowledge and representations inside LLMs

Abstract (arXiv)

Mechanistic interpretability identifies internal circuits responsible for model behaviors, yet translating these findings into human-understandable explanations remains an open problem. We present a pipeline that bridges circuit-level analysis and natural language explanations by (i) identifying causally important attention heads via activation patching, (ii) generating explanations using both template-based and LLM-based methods, and (iii) evaluating faithfulness using ERASER-style metrics adapted for circuit-level attribution. We evaluate on the Indirect Object Identification (IOI) task in GPT-2 Small (124M parameters), identifying six attention heads accounting for 61.4% of the logit difference. Our circuit-based explanations achieve 100% sufficiency but only 22% comprehensiveness, revealing distributed backup mechanisms. LLM-generated explanations outperform template baselines by 64% on quality metrics. We find no correlation (r = 0.009) between model confidence and explanation faithfulness, and identify three failure categories explaining when explanations diverge from mechanisms.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Knowledge and representations inside LLMs, one of 13 directions in Mechanistic interpretability. Builds on 8 of its 13 references that are in the map. First posted on arXiv 13 Feb 2026.

Builds on

8 of 13 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024Towards Faithful Natural Language Explanations: A Study Using Activation Patching in Large Language Models18 Oct

Other references · 7

  1. 2022Interpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 smallcited by 1,165
  2. 2019Attention is not Explanationcited by 929
  3. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 724
  4. 2019Attention is not not Explanationcited by 636
  5. 2019ERASER: A Benchmark to Evaluate Rationalized NLP Modelscited by 530
  6. 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
  7. 2021Causal Abstractions of Neural Networkscited by 422

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Explainability of Large Language Models: Opportunities and Challenges toward Generating Trustworthy Explanations
  2. 2026Rethinking Circuit Evaluation: Do Circuits Explain Model Errors?
  3. 2026Mechanistic Data Attribution: Tracing the Training Origins of Interpretable LLM Units
  4. 2026Are We Recovering Mechanisms? Objective-Level Recovery Gaps in Mechanistic Interpretability
  5. 2026ICE: Intervention-Consistent Explanation Evaluation with Statistical Grounding for LLMs
  6. 2024Towards Uncovering How Large Language Model Works: An Explainability Perspective
  7. 2025Toward a Theory of Generalizability in LLM Mechanistic Interpretability Research
  8. 2025Evaluating Explanations: An Explanatory Virtues Framework for Mechanistic Interpretability -- The Strange Science Part I.ii