mlsanityData to 7 Oct 2026

Reasoning Errors Have a Region and a Direction in the Residual-Stream Trajectory of LLMs

arXiv 2608.05660 ↗First posted 6 Aug 2026cs.LG cs.CLcited by 0

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

As language models are increasingly used for tasks that require verifiable reasoning, reliably distinguishing sound reasoning from flawed reasoning has become an important practical problem. Recent trajectory-based methods seek this signal in layerwise residual-stream displacements, which capture how representations change while attenuating some stable, token-specific information. However, displacement omits the state from which an update originates, whereas restoring the full state risks reintroducing shortcut-prone information. We identify this trade-off and propose a three-stream detector that combines motion with two restricted views of location. A coarse region reader based on vector quantization and a fine direction reader over normalized multi-layer states. This design restores enough state context to interpret the motion without returning to full-state probing. On reasoning benchmarks unseen during training, our method improves selection accuracy by up to 12% over the displacement-only state of the art and 21% over single-layer probing baselines. Although trained only on reasoning benchmarks, it also reads factual completion and fact verification, ahead of every detector we compare against, which places the signal on correctness rather than on a kind of reasoning. Ablations further show that motion, region, and direction provide complementary signals. These results suggest that reasoning validity is better read from state-conditioned motion than from either static states or decontextualized trajectories alone.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 18 of its 46 references that are in the map. First posted on arXiv 6 Aug 2026.

Builds on

18 of 46 references.

References · 18

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2024Qwen2.5 Technical Reportcited by 4,713
  4. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  5. 2022Toy Models of Superpositioncited by 928
  6. 2023Steering Language Models With Activation Engineeringcited by 824
  7. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 733
  8. 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 679
  9. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
  10. 2023Eliciting Latent Predictions from Transformers with the Tuned Lenscited by 579
  11. 2023The emergence of clusters in self-attention dynamicscited by 133
  12. 2019Understanding and Improving Transformer From a Multi-Particle Dynamic System Point of Viewcited by 108
6 more
  1. 2025Language Models are Injective and Hence Invertiblecited by 31
  2. 2025Large Language Models are overconfident and amplify human biascited by 30
  3. 2025The Open Proof Corpus: A Large-Scale Study of LLM-Generated Mathematical Proofscited by 27
  4. 2025The Geometry of Reasoning: Flowing Logics in Representation Spacecited by 21
  5. 2025I Predict Therefore I Am: Is Next Token Prediction Enough to Learn Human-Interpretable Concepts from Data?cited by 13
  6. 2025Transformer Dynamics: A neuroscientific approach to interpretability of large language modelscited by 11

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Search for Truth from Reasoning: A Dynamic Representation Editing Framework for Steering LLM Trajectories
  2. 2026The Error You See Is Not the Error You Made: Progression-aware Reasoning Origin for Reasoning Error Localization
  3. 2026Truth as a Trajectory: What Internal Representations Reveal About Large Language Model Reasoning
  4. 2026Revisiting Complete Reasoning Traces for Post-Training
  5. 2026BLADE: Boundary-Expanded and Layer-Adaptive Dynamic Exit for Efficient LLM Reasoning
  6. 2026Silent Failures in Quantized LLM Reasoning: A Taxonomy-Based Analysis of Hollow Convergence and Failure Mode Shifts
  7. 2026Confidence Geometry Reveals Trace-Level Correctness in Large Language Model Reasoning
  8. 2026Invariant Reasoning Directions in Latent Trajectories of Language Models