mlsanityData to 14 Oct 2026

Interpretability without actionability: mechanistic methods cannot correct language model errors despite near-perfect internal representations

arXiv 2603.18353 ↗First posted 18 Mar 2026cs.AIcited by 17

On the map: Mechanistic interpretability → Interpreting biomedical and science models

Abstract (arXiv)

Language models encode task-relevant knowledge in internal representations that far exceeds their output performance, but whether mechanistic interpretability methods can bridge this knowledge-action gap has not been systematically tested. We compared four mechanistic interpretability methods -- concept bottleneck steering (Steerling-8B), sparse autoencoder feature steering, logit lens with activation patching, and linear probing with truthfulness separator vector steering (Qwen 2.5 7B Instruct) -- for correcting false-negative triage errors using 400 physician-adjudicated clinical vignettes (144 hazards, 256 benign). Linear probes discriminated hazardous from benign cases with 98.2% AUROC, yet the model's output sensitivity was only 45.1%, a 53-percentage-point knowledge-action gap. Concept bottleneck steering corrected 20% of missed hazards but disrupted 53% of correct detections, indistinguishable from random perturbation (p=0.84). SAE feature steering produced zero effect despite 3,695 significant features. TSV steering at high strength corrected 24% of missed hazards while disrupting 6% of correct detections, but left 76% of errors uncorrected. Current mechanistic interpretability methods cannot reliably translate internal knowledge into corrected outputs, with implications for AI safety frameworks that assume interpretability enables effective error correction.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Placed in Interpreting biomedical and science models, one of 13 directions in Mechanistic interpretability. Builds on 12 of its 31 references that are in the map. First posted on arXiv 18 Mar 2026.

Builds on

12 of 31 references.

References · 12

  1. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  2. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  3. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  4. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  5. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  6. 2020Concept Bottleneck Modelscited by 1,032
  7. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 767
  8. 2023Towards Automated Circuit Discovery for Mechanistic Interpretabilitycited by 724
  9. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
  10. 2023A Closer Look at the Intervention Procedure of Concept Bottleneck Modelscited by 64
  11. 2024Concept Bottleneck Large Language Modelscited by 43
  12. 2026Decomposing Physician Disagreement in HealthBenchcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 17.

17 papers.

Sort:

By area:

  1. 2026Where Do Apparent LLM Clinical Triage Failures Arise? Localizing the Multiple-Choice Format Effectcited by 4
  2. 2026Steering Without Breaking: Mechanistically Informed Interventions for Discrete Diffusion Language Modelscited by 3
  3. 2026Decodable but Not Corrected by Fixed Residual-Stream Linear Steering: Evidence from Medical LLM Failure Regimescited by 2
  4. 2026Decodable Is Not Grounded: A Vision-Ablation Arbiter for VLM Spatial Reasoningcited by 2
  5. 2026Syntax Without Semantics: Teaching Large Language Models to Code in an Unseen Languagecited by 1
  6. 2026Causal Tongue-Tie: LLMs Can Encode Causal Direction, But Their Yes/No Outputs Fail to Expresscited by 1
  7. 2026Confidence and Calibration of Activation Oracles for Reliable Interpretation of Language Model Internalscited by 1
  8. 2026Legible Failures: Detecting and Repairing In-Context Binding Errorscited by 1
  9. 2026Generative Interpretability via Scalable Neuro-Symbolic Modelscited by 1
  10. 2026Readable Before Actionable: Causal Tracing of Indirect Prompt Injectioncited by 1
  11. 2026The Open-Box Fallacy: Why AI Deployment Needs a Calibrated Verification Regimecited by 0
  12. 2026Relational Rank Geometry in Transformers: Detecting and Steering Hidden-State Relation Framescited by 0
5 more
  1. 2026Validation-Gated Causal Interventions for Interpreting High-Stakes Large Language Model Behavior: A Case Study in Suicidality Detectioncited by 0
  2. 2026Reality Monitoring in Large Language Models: Self-Knowledge That Transforms with Conversation Memorycited by 0
  3. 2026How Do Language Models Choose Between Context and Memory?cited by 0
  4. 2026Where Does Jagged Competence Come From?cited by 0
  5. 2026Principled Under Pressure: Post-Training Decides Whether LLMs Act on Their Own Moral Judgmentcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Locate, Steer, and Improve: A Practical Survey of Actionable Mechanistic Interpretability in Large Language Models
  2. 2024Towards Unifying Interpretability and Control: Evaluation via Intervention
  3. 2024Mechanistic?
  4. 2026Causally Grounded Mechanistic Interpretability for LLMs with Faithful Natural-Language Explanations
  5. 2026When the Coffee Feature Activates on Coffins: An Analysis of Feature Extraction and Steering for Mechanistic Interpretability
  6. 2026Scaling Inherently Interpretable Language Models
  7. 2026Make Mechanistic Interpretability Auditable: A Call to Develop Guidelines via Continuous Collaborative Reviewing
  8. 2026MechELK: A Mechanistic Interpretability Framework for Eliciting Latent Knowledge in Large Language Models