# LLM Reasoning as Trajectories: Step-Specific Representation Geometry and Correctness Signals

arXiv: https://arxiv.org/abs/2604.05655 · HTML: https://mlsanity.com/paper/2604.05655/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning.
- Builds on 25 of its 30 references that are in the map.
- First posted on arXiv 7 Apr 2026.

## Placed in

- LLM reasoning / [Test-time compute and efficient thinking](https://mlsanity.com/lr/test-time-compute-and-efficient-thinking/)

## Builds on (most cited first, 25 of 25)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [s1: Simple test-time scaling](https://mlsanity.com/paper/2501.19393/) (2025)
- [Steering Language Models With Activation Engineering](https://mlsanity.com/paper/2308.10248/) (2023)
- [Measuring Faithfulness in Chain-of-Thought Reasoning](https://mlsanity.com/paper/2307.13702/) (2023)
- [Towards Best Practices of Activation Patching in Language Models: Metrics and Methods](https://mlsanity.com/paper/2309.16042/) (2023)
- [Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters](https://mlsanity.com/paper/2212.10001/) (2022)
- [Chain-of-Thought Reasoning In The Wild Is Not Always Faithful](https://mlsanity.com/paper/2503.08679/) (2025)
- [Reasoning Models Know When They're Right: Probing Hidden States for Self-Verification](https://mlsanity.com/paper/2504.05419/) (2025)
- [Mind Your Step (by Step): Chain-of-Thought can Reduce Performance on Tasks where Thinking Makes Humans Worse](https://mlsanity.com/paper/2410.21333/) (2024)
- [Wait, We Don't Need to "Wait"! Removing Thinking Tokens Improves Reasoning Efficiency](https://mlsanity.com/paper/2506.08343/) (2025)
- [How to think step-by-step: A mechanistic understanding of chain-of-thought reasoning](https://mlsanity.com/paper/2402.18312/) (2024)
- [Does Thinking More always Help? Mirage of Test-Time Scaling in Reasoning Models](https://mlsanity.com/paper/2506.04210/) (2025)
- [Demystifying Reasoning Dynamics with Mutual Information: Thinking Tokens are Information Peaks in LLM Reasoning](https://mlsanity.com/paper/2506.02867/) (2025)
- [AlphaOne: Reasoning Models Thinking Slow and Fast at Test Time](https://mlsanity.com/paper/2505.24863/) (2025)
- [Landscape of Thoughts: Visualizing the Reasoning Process of Large Language Models](https://mlsanity.com/paper/2503.22165/) (2025)
- [The Geometry of Reasoning: Flowing Logics in Representation Space](https://mlsanity.com/paper/2510.09782/) (2025)
- [Valence-Arousal Subspace in LLMs: Circular Emotion Geometry and Multi-Behavioral Control](https://mlsanity.com/paper/2604.03147/) (2026)
- [Paraphrase and Solve: Exploring and Exploiting the Impact of Surface Form on Mathematical Reasoning in Large Language Models](https://mlsanity.com/paper/2404.11500/) (2024)
- [The Geometry of Self-Verification in a Task-Specific Reasoning Model](https://mlsanity.com/paper/2504.14379/) (2025)
- [CLUE: Non-parametric Verification from Experience via Hidden-State Clustering](https://mlsanity.com/paper/2510.01591/) (2025)
- [Test-Time Scaling in Reasoning Models Is Not Effective for Knowledge-Intensive Tasks Yet](https://mlsanity.com/paper/2509.06861/) (2025)
- [LLM Microscope: What Model Internals Reveal About Answer Correctness and Context Utilization](https://mlsanity.com/paper/2510.04013/) (2025)
- [REMA: A Unified Reasoning Manifold Framework for Interpreting Large Language Model](https://mlsanity.com/paper/2509.22518/) (2025)

## Built on by (most cited first, 24 of 24)

- [Latent Thought Flow: Efficient Latent Reasoning in Large Language Models](https://mlsanity.com/paper/2606.16222/) (2026)
- [SliceGraph: Mapping Process Isomers in Multi-Run Chain-of-Thought Reasoning](https://mlsanity.com/paper/2605.14619/) (2026)
- [Reasoning Models Don't Just Think Longer, They Move Differently](https://mlsanity.com/paper/2605.15454/) (2026)
- [AutoCRAT: Within-trajectory Joint Control of Stochasticity and Compute for LLM Reasoning](https://mlsanity.com/paper/2608.29988/) (2026)
- [Semantic Step Prediction: Multi-Step Latent Forecasting in LLM Reasoning Trajectories via Step Sampling](https://mlsanity.com/paper/2604.18464/) (2026)
- [Uncovering the Representation Geometry of Minimal Cores in Overcomplete Reasoning Traces](https://mlsanity.com/paper/2605.14358/) (2026)
- [Geometric Signatures of Reasoning: A Spectral Perspective on Task Hardness](https://mlsanity.com/paper/2607.01571/) (2026)
- [It's the Problem, Not the Path: Budget and Difficulty Confounds in LLM Reasoning Trajectories](https://mlsanity.com/paper/2609.03436/) (2026)
- [Erased, Rerouted, or Rescaled? Post-Training and the Causal Quotient of a Language Model's Belief State](https://mlsanity.com/paper/2610.05292/) (2026)
- [GeoFaith: A Spatio-Temporal Dual View of Faithful Chain-of-Thought](https://mlsanity.com/paper/2605.26893/) (2026)
- [Right Makes Might: Aligning Verified Hidden States Empowers RL Reasoning](https://mlsanity.com/paper/2606.03234/) (2026)
- [LoRi: Low-Rank Distillation for Implicit Reasoning](https://mlsanity.com/paper/2606.05315/) (2026)
- [What Process Evaluation of Coding Agents Actually Measures: Action, Task, and Step Are Three Different Levels](https://mlsanity.com/paper/2608.22960/) (2026)
- [From Concentration to Differentiation and Back: Routing Effective Rank in MoE Reasoning Cohorts](https://mlsanity.com/paper/2609.06403/) (2026)
- [Do Agents Know When They Succeed? Calibrating Agent Confidence from Internal Representations](https://mlsanity.com/paper/2609.09448/) (2026)
- [State of Thought Enables Endogenous Reasoning](https://mlsanity.com/paper/2609.16055/) (2026)
- [Reasoning-Preserving Fine-Tuning of Post-RL LLMs with Null-Basis LoRA](https://mlsanity.com/paper/2609.25618/) (2026)
- [Logical subspace in LLMs](https://mlsanity.com/paper/2609.32907/) (2026)
- [Rethinking Reasoning Paths as Phase-Structured Trajectories](https://mlsanity.com/paper/2609.36461/) (2026)
- [Hesitation Has a Geometry: Entropy-Trained Hyperbolic Probes for Sparse Activation Steering](https://mlsanity.com/paper/2610.02391/) (2026)
- [Spend Teacher Tokens Where They Matter: Success-Referenced On-Policy Distillation](https://mlsanity.com/paper/2610.02678/) (2026)
- [ThinkFuse: Trajectory-Aware Test-Time Fusion for Small Reasoning Models](https://mlsanity.com/paper/2610.07803/) (2026)
- [From Candidate Tokens to Final Outcomes: Decoding Relative Long-Horizon Candidate Value from Reasoning Model Representations](https://mlsanity.com/paper/2610.15412/) (2026)
- [Divergent Paths: Learning an Outcome-Predictive Representation of LLM Agent Trajectories](https://mlsanity.com/paper/2610.19614/) (2026)

## Abstract (arXiv)

This work characterizes large language models' chain-of-thought generation as a structured trajectory through representation space. We show that mathematical reasoning traverses functionally ordered, step-specific subspaces that become increasingly separable with layer depth. This structure already exists in base models, while reasoning training primarily accelerates convergence toward termination-related subspaces rather than introducing new representational organization. While early reasoning steps follow similar trajectories, correct and incorrect solutions diverge systematically at late stages. This late-stage divergence enables mid-reasoning prediction of final-answer correctness with ROC-AUC up to 0.87. Furthermore, we introduce trajectory-based steering, an inference-time intervention framework that enables reasoning correction and length control based on derived ideal trajectories. Together, these results establish reasoning trajectories as a geometric lens for interpreting, predicting, and controlling LLM reasoning behavior.