mlsanityData to 14 Oct 2026

Linear Probes Detect Task Format, Not Reasoning Mode in Language Model Hidden States

arXiv 2606.02907 ↗First posted 1 Jun 2026cs.CL cs.AIcited by 7

On the map: LLM reasoning → Reasoning benchmarks and evaluation

also in 1 more
Abstract (arXiv)

Linear probing of large language model (LLM) hidden states is widely used to claim that models learn distinct representations for different reasoning types. We test this by probing Qwen3-14B on three benchmarks spanning the classical trichotomy: LogiQA 2.0 (deductive), ARC-Challenge (inductive), and αNLI (abductive). At layer 32 of 40, linear probes achieve 100% cross-validated accuracy with well-separated geometry (intrinsic dimensionalities: 20.6, 28.5, 33.6; convex hull contamination ≤1.5%). However, this separation is entirely driven by format confounds. Residualizing source identity, option count, and response length reduces accuracy to chance. Trace-anchor similarity indicates largely shared reasoning across tasks (42.5% agreement vs. 33.3% chance), and causal steering with random controls (n=20) shows no functional link between geometry and reasoning mode (p=0.286). Thus, high probe accuracy reflects task format rather than computational structure, motivating routine format deconfounding in mechanistic interpretability.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 22 of its 32 references that are in the map. First posted on arXiv 1 Jun 2026.

Builds on

22 of 32 references.

Closest ancestors · 6

Same direction, newest first

  1. 2026The Reasoning Trap -- Logical Reasoning as a Mechanistic Pathway to Situational Awareness10 Mar
  2. 2025Qwen3 Technical Report14 May
  3. 2024Reasoning in Large Language Models: A Geometric Perspective2 Jul
  4. 2022Towards Reasoning in Large Language Models: A Survey20 Dec
  5. 2022Emergent Analogical Reasoning in Large Language Models19 Dec
  6. 2022FOLIO: Natural Language Reasoning with First-Order Logic2 Sep

Other references · 16

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  5. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,886
  6. 2023Mistral 7Bcited by 3,164
  7. 2022Locating and Editing Factual Associations in GPTcited by 2,691
  8. 2016Understanding intermediate layers using linear classifier probescited by 1,353
  9. 2023Representation Engineering: A Top-Down Approach to AI Transparencycited by 1,336
  10. 2023Inference-Time Intervention: Eliciting Truthful Answers from a Language Modelcited by 1,302
  11. 2023Steering Language Models With Activation Engineeringcited by 922
  12. 2023Progress measures for grokking via mechanistic interpretabilitycited by 839
4 more
  1. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 712
  2. 2019Abductive Commonsense Reasoningcited by 301
  3. 2024Large Language Models for Mathematical Reasoning: Progresses and Challengescited by 284
  4. 2026When AI Benchmarks Plateau: A Systematic Study of Benchmark Saturationcited by 33

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 7.

7 papers.

  1. 2026The Entanglement Wall: Activation-Space Probes as Risk Detectors, Not Context Adjudicatorscited by 2
  2. 2026Decodability is Not Causality: Dissociating Probe Readouts from Behavioral Drivers via SAE Decompositioncited by 2
  3. 2026When Decodability Is Not Enough: Logical Validity Representations, Behavioral Dissociation, and Causal Tests in Language Modelscited by 1
  4. 2026Calibration of Structured Ignorance Certificates for Diagnosing Unknown Unknowns in Reasoning Modelscited by 0
  5. 2026Creative Collision: Directorial Persona Steering and Competition in Large Language Modelscited by 0
  6. 2026From Interpretability to Control: Insights from Six Years of the TrustNLP Workshopcited by 0
  7. 2026Falsehood and Impossibility Are Different Directions in an AI's Representation of Languagecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Monitoring Latent World States in Language Models with Propositional Probes
  2. 2025Large Language Models Encode Semantics and Alignment in Linearly Separable Representations
  3. 2026H-Probes: Extracting Hierarchical Structures From Latent Representations of Language Models
  4. 2026Detecting Hidden Chain-of-Thought in Large Language Models with Linguistic, Behavioral, and Mechanistic Indicators
  5. 2025CLUE: Non-parametric Verification from Experience via Hidden-State Clustering
  6. 2026LLM Reasoning Is Latent, Not the Chain of Thought
  7. 2026The Tell-Tale Norm: ℓ2 Magnitude as a Signal for Reasoning Dynamics in Large Language Models
  8. 2026Beyond Language: Format-Agnostic Reasoning Subspaces in Large Language Models