# Interpretability without actionability: mechanistic methods cannot correct language model errors despite near-perfect internal representations

arXiv: https://arxiv.org/abs/2603.18353 · HTML: https://mlsanity.com/paper/2603.18353/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Placed in Interpreting biomedical and science models, one of 13 directions in Mechanistic interpretability.
- Builds on 12 of its 31 references that are in the map.
- First posted on arXiv 18 Mar 2026.

## Placed in

- Mechanistic interpretability / [Interpreting biomedical and science models](https://mlsanity.com/mechinterp/interpreting-biomedical-and-science-models/)

## Builds on (most cited first, 12 of 12)

- [Large Language Models Encode Clinical Knowledge](https://mlsanity.com/paper/2212.13138/) (2022)
- [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [Representation Engineering: A Top-Down Approach to AI Transparency](https://mlsanity.com/paper/2310.01405/) (2023)
- [Inference-Time Intervention: Eliciting Truthful Answers from a Language Model](https://mlsanity.com/paper/2306.03341/) (2023)
- [Concept Bottleneck Models](https://mlsanity.com/paper/2007.04612/) (2020)
- [Discovering Latent Knowledge in Language Models Without Supervision](https://mlsanity.com/paper/2212.03827/) (2022)
- [Towards Automated Circuit Discovery for Mechanistic Interpretability](https://mlsanity.com/paper/2304.14997/) (2023)
- [The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets](https://mlsanity.com/paper/2310.06824/) (2023)
- [A Closer Look at the Intervention Procedure of Concept Bottleneck Models](https://mlsanity.com/paper/2302.14260/) (2023)
- [Concept Bottleneck Large Language Models](https://mlsanity.com/paper/2412.07992/) (2024)
- [Decomposing Physician Disagreement in HealthBench](https://mlsanity.com/paper/2602.22758/) (2026)

## Built on by (most cited first, 17 of 17)

- [Where Do Apparent LLM Clinical Triage Failures Arise? Localizing the Multiple-Choice Format Effect](https://mlsanity.com/paper/2605.29889/) (2026)
- [Steering Without Breaking: Mechanistically Informed Interventions for Discrete Diffusion Language Models](https://mlsanity.com/paper/2605.10971/) (2026)
- [Decodable but Not Corrected by Fixed Residual-Stream Linear Steering: Evidence from Medical LLM Failure Regimes](https://mlsanity.com/paper/2605.05715/) (2026)
- [Decodable Is Not Grounded: A Vision-Ablation Arbiter for VLM Spatial Reasoning](https://mlsanity.com/paper/2606.31257/) (2026)
- [Syntax Without Semantics: Teaching Large Language Models to Code in an Unseen Language](https://mlsanity.com/paper/2605.15607/) (2026)
- [Causal Tongue-Tie: LLMs Can Encode Causal Direction, But Their Yes/No Outputs Fail to Express](https://mlsanity.com/paper/2605.25891/) (2026)
- [Confidence and Calibration of Activation Oracles for Reliable Interpretation of Language Model Internals](https://mlsanity.com/paper/2605.26045/) (2026)
- [Legible Failures: Detecting and Repairing In-Context Binding Errors](https://mlsanity.com/paper/2609.11216/) (2026)
- [Generative Interpretability via Scalable Neuro-Symbolic Models](https://mlsanity.com/paper/2609.13529/) (2026)
- [Readable Before Actionable: Causal Tracing of Indirect Prompt Injection](https://mlsanity.com/paper/2610.05295/) (2026)
- [The Open-Box Fallacy: Why AI Deployment Needs a Calibrated Verification Regime](https://mlsanity.com/paper/2605.10601/) (2026)
- [Relational Rank Geometry in Transformers: Detecting and Steering Hidden-State Relation Frames](https://mlsanity.com/paper/2605.29634/) (2026)
- [Validation-Gated Causal Interventions for Interpreting High-Stakes Large Language Model Behavior: A Case Study in Suicidality Detection](https://mlsanity.com/paper/2606.21078/) (2026)
- [Reality Monitoring in Large Language Models: Self-Knowledge That Transforms with Conversation Memory](https://mlsanity.com/paper/2607.23927/) (2026)
- [How Do Language Models Choose Between Context and Memory?](https://mlsanity.com/paper/2609.00753/) (2026)
- [Where Does Jagged Competence Come From?](https://mlsanity.com/paper/2610.03831/) (2026)
- [Principled Under Pressure: Post-Training Decides Whether LLMs Act on Their Own Moral Judgment](https://mlsanity.com/paper/2610.08670/) (2026)

## Abstract (arXiv)

Language models encode task-relevant knowledge in internal representations that far exceeds their output performance, but whether mechanistic interpretability methods can bridge this knowledge-action gap has not been systematically tested. We compared four mechanistic interpretability methods -- concept bottleneck steering (Steerling-8B), sparse autoencoder feature steering, logit lens with activation patching, and linear probing with truthfulness separator vector steering (Qwen 2.5 7B Instruct) -- for correcting false-negative triage errors using 400 physician-adjudicated clinical vignettes (144 hazards, 256 benign). Linear probes discriminated hazardous from benign cases with 98.2% AUROC, yet the model's output sensitivity was only 45.1%, a 53-percentage-point knowledge-action gap. Concept bottleneck steering corrected 20% of missed hazards but disrupted 53% of correct detections, indistinguishable from random perturbation (p=0.84). SAE feature steering produced zero effect despite 3,695 significant features. TSV steering at high strength corrected 24% of missed hazards while disrupting 6% of correct detections, but left 76% of errors uncorrected. Current mechanistic interpretability methods cannot reliably translate internal knowledge into corrected outputs, with implications for AI safety frameworks that assume interpretability enables effective error correction.