mlsanityData to 14 Oct 2026

Decodable but Not Corrected by Fixed Residual-Stream Linear Steering: Evidence from Medical LLM Failure Regimes

arXiv 2605.05715 ↗First posted 7 May 2026cs.AI cs.CL cs.LGcited by 2

Not on the map. Why?

Abstract (arXiv)

Can linearly decodable failure signals in LLM hidden states be leveraged to correct those failures? We investigate this classification-correction gap via Overthinking (OT)--a stable behavioral regime (Jaccard >= 0.81, 94% inter-annotator agreement) in medical QA where models answer correctly under resampling yet fail in extended chain-of-thought. OT is linearly decodable at 71.6% balanced accuracy (p < 10^{-16}). Yet five families of fixed linear steering (29 configurations, n=1,273) all yield Delta ~= 0, with identical null results cross-architecture (Qwen2.5-7B) and cross-domain (MMLU-STEM). Three convergent lines of evidence suggest representational entanglement: the OT direction has 85-88% overlap with task-critical computation (specificity ratio <= 0.152); non-targeted shared-direction steering damages accuracy (-12.1pp); and LEACE concept erasure damages accuracy (-3.6pp, p=0.01), while 10 random erasures produce Delta=+0.3pp. The per-instance probe-steering correlation is r=-0.002 (p=0.97). Positively, the same probe enables selective abstention (held-out AUROC=0.610, exceeding all five uncertainty baselines, p=0.009): decodable failure structure supports post-generation reliability estimation even when the fixed linear steering family cannot exploit it for correction.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 50 of its 73 references that are in the map. First posted on arXiv 7 May 2026.

Builds on

50 of 73 references.

References · 50

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  3. 2017On Calibration of Modern Neural Networkscited by 4,960
  4. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  5. 2023Let's Verify Step by Stepcited by 3,789
  6. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  7. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  8. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  9. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  10. 2022Large Language Models Encode Clinical Knowledgecited by 1,830
  11. 2023Sparse Autoencoders Find Highly Interpretable Features in Language Modelscited by 1,420
  12. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
38 more
  1. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  2. 2025s1: Simple test-time scalingcited by 1,341
  3. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  4. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  5. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
  6. 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
  7. 2024Refusal in Language Models Is Mediated by a Single Directioncited by 1,032
  8. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 1,017
  9. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 974
  10. 2023Steering Language Models With Activation Engineeringcited by 922
  11. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 876
  12. 2017Selective Classification for Deep Neural Networkscited by 852
  13. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 767
  14. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
  15. 2019Designing and Interpreting Probes with Control Taskscited by 640
  16. 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMscited by 570
  17. 2020Null It Out: Guarding Protected Attributes by Iterative Nullspace Projectioncited by 455
  18. 2023Emergent Linear Representations in World Models of Self-Supervised Sequence Modelscited by 319
  19. 2023LEACE: Perfect linear concept erasure in closed formcited by 267
  20. 2024To CoT or not to CoT? Chain-of-thought helps mainly on math and symbolic reasoningcited by 263
  21. 2023Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Modelscited by 243
  22. 2023Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representationscited by 209
  23. 2023Linearity of Relation Decoding in Transformer Language Modelscited by 200
  24. 2024ReFT: Representation Finetuning for Language Modelscited by 197
  25. 2023Localizing Model Behavior with Path Patchingcited by 195
  26. 2025Thoughts Are All Over the Place: On the Underthinking of o1-Like LLMscited by 163
  27. 2025The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independencecited by 115
  28. 2023Improving Activation Steering in Language Models with Mean-Centringcited by 68
  29. 2025Steering off Course: Reliability Challenges in Steering Language Modelscited by 34
  30. 2025Mitigating Content Effects on Reasoning in Language Models through Fine-Grained Activation Steeringcited by 18
  31. 2026Interpretability without actionability: mechanistic methods cannot correct language model errors despite near-perfect internal representationscited by 17
  32. 2026Steered LLM Activations are Non-Surjectivecited by 8
  33. 2026ASA: Backbone-Training-Free Representation Engineering for Tool-Calling Agentscited by 6
  34. 2026Post-Hoc Reasoning in Chain of Thought: Decoding and Steering Pre-Committed Answerscited by 6
  35. 2026Steerable but Not Decodable: Function Vectors Operate Beyond the Logit Lenscited by 6
  36. 2026Endogenous Resistance to Activation Steering in Language Modelscited by 5
  37. 2026The Cylindrical Representation Hypothesis for Language Model Steeringcited by 5
  38. 2026Predicting Where Steering Vectors Succeedcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Reproducing and Stress-Testing Two Approaches to LLM Reasoning Reliability: Test-Time Probability Aggregation and Logic-Representation Editingcited by 0
  2. 2026SelfCue: Making a 3D CT Report Generator Say What It Already Knowscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Silent Failures in Quantized LLM Reasoning: A Taxonomy-Based Analysis of Hollow Convergence and Failure Mode Shifts
  2. 2025Black-Box Behavioral Distillation Breaks Safety Alignment in Medical LLMs
  3. 2026Adapter Merging Reactivates Latent Reasoning Traces: A Mechanism Analysis
  4. 2026Decodability is Not Causality: Dissociating Probe Readouts from Behavioral Drivers via SAE Decomposition
  5. 2026ROM: Real-time Overthinking Mitigation via Streaming Detection and Intervention
  6. 2026From Signal Degradation to Computation Collapse: Uncovering the Two Failure Modes of LLM Quantization
  7. 2026Are We Recovering Mechanisms? Objective-Level Recovery Gaps in Mechanistic Interpretability
  8. 2026Overconfidence and Calibration in Medical VQA: Empirical Findings and Hallucination-Aware Mitigation