Rethinking Reasoning Paths as Phase-Structured Trajectories
On the map: LLM reasoning → Test-time compute and efficient thinking also fits: Knowledge- and tool-augmented reasoning
Abstract (arXiv)
Large language models often improve problem-solving performance by generating multi-step reasoning paths, yet how to analyze the hidden states along these paths remains unclear. Existing approaches typically assign each intermediate state the final-answer correctness label and train probes across heterogeneous questions. We argue that this protocol obscures reasoning dynamics in two ways: (1) correctness prediction can exploit question-level variation rather than path quality, and (2) states aligned by absolute step indices may correspond to different functional phases of reasoning. In this work, we propose to view reasoning paths as phase-structured trajectories within fixed questions. We instantiate this view as PAIR, short for Phase-Aligned Intra-question Reasoning. PAIR samples multiple trajectories for each question, maps variable-length paths into shared relative phases based on normalized trajectory progress, and compares successful and unsuccessful trajectories only within the same question and phase. This yields phase-specific path-quality directions that better isolate path-quality signals from question-level variation. Empirically, we find that standard across-question correctness probes lose much of their predictive power under within-question evaluation, suggesting that these probes partly rely on question-level information. PAIR improves within-question trajectory ranking and Best-of-N trajectory selection across models and benchmarks. Phase-wise steering further shows that the learned directions can change generation outcomes, providing causal evidence that they capture trajectory-relevant information.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 15 of its 24 references that are in the map. First posted on arXiv 29 Sep 2026.
Builds on
15 of 24 references.
Closest ancestors · 2
Same direction, newest first
- 2026LLM Reasoning as Trajectories: Step-Specific Representation Geometry and Correctness Signals7 Apr
- 2025Reasoning Models Know When They're Right: Probing Hidden States for Self-Verification7 Apr
Other references · 13
- 2021Training Verifiers to Solve Math Word Problemscited by 10,109
- 2025Qwen3 Technical Reportcited by 8,820
- 2021Evaluating Large Language Models Trained on Codecited by 8,320
- 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
- 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
- 2023Let's Verify Step by Stepcited by 3,789
- 2025Gemma 3 Technical Reportcited by 1,824
- 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
- 2023Steering Llama 2 via Contrastive Activation Additioncited by 1,085
- 2023The Linear Representation Hypothesis and the Geometry of Large Language Modelscited by 716
- 2026Reasoning Theater: Disentangling Model Beliefs from Chain-of-Thoughtcited by 40
- 2025LLM Microscope: What Model Internals Reveal About Answer Correctness and Context Utilizationcited by 11
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Answer-Distribution Trajectories: A Stochastic-Dynamics View of LLM Reasoning
- 2026Probing the Trajectories of Reasoning Traces in Large Language Models
- 2023Towards A Unified View of Answer Calibration for Multi-Step Reasoning
- 2025Follow the Path: Reasoning over Knowledge Graph Paths to Improve Large Language Model Factuality
- 2026The Path Matters: Evaluating Small Language Models Beyond Answer Accuracy in KGQA
- 2026Invariant Reasoning Directions in Latent Trajectories of Language Models
- 2026Confidence Geometry Reveals Trace-Level Correctness in Large Language Model Reasoning
- 2025Tracing the Traces: Latent Temporal Signals for Efficient and Accurate Reasoning