# Truth as a Trajectory: What Internal Representations Reveal About Large Language Model Reasoning

arXiv: https://arxiv.org/abs/2603.01326 · HTML: https://mlsanity.com/paper/2603.01326/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning.
- Builds on 14 of its 36 references that are in the map.
- First posted on arXiv 1 Mar 2026.

## Placed in

- LLM reasoning / [Reasoning benchmarks and evaluation](https://mlsanity.com/lr/reasoning-benchmarks-and-evaluation/)
- Mixture-of-experts / [MoE large language models](https://mlsanity.com/moe/moe-large-language-models/)
- Mechanistic interpretability / [Knowledge and representations inside LLMs](https://mlsanity.com/mechinterp/knowledge-and-representations-inside-llms/)

## Builds on (most cited first, 14 of 14)

- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [Representation Engineering: A Top-Down Approach to AI Transparency](https://mlsanity.com/paper/2310.01405/) (2023)
- [Toy Models of Superposition](https://mlsanity.com/paper/2209.10652/) (2022)
- [Discovering Latent Knowledge in Language Models Without Supervision](https://mlsanity.com/paper/2212.03827/) (2022)
- [The Linear Representation Hypothesis and the Geometry of Large Language Models](https://mlsanity.com/paper/2311.03658/) (2023)
- [The emergence of clusters in self-attention dynamics](https://mlsanity.com/paper/2305.05465/) (2023)
- [Understanding and Improving Transformer From a Multi-Particle Dynamic System Point of View](https://mlsanity.com/paper/1906.02762/) (2019)
- [Steering Language Model Refusal with Sparse Autoencoders](https://mlsanity.com/paper/2411.11296/) (2024)
- [A Comprehensive Survey on the Trustworthiness of Large Language Models in Healthcare](https://mlsanity.com/paper/2502.15871/) (2025)
- [The Geometry of Reasoning: Flowing Logics in Representation Space](https://mlsanity.com/paper/2510.09782/) (2025)
- [I Predict Therefore I Am: Is Next Token Prediction Enough to Learn Human-Interpretable Concepts from Data?](https://mlsanity.com/paper/2503.08980/) (2025)
- [Transformer Dynamics: A neuroscientific approach to interpretability of large language models](https://mlsanity.com/paper/2502.12131/) (2025)
- [Curved Inference: Concern-Sensitive Geometry in Large Language Model Residual Streams](https://mlsanity.com/paper/2507.21107/) (2025)
- [Latent Trajectory Dynamics in Large Language Models: A Manifold Evolution Framework with Empirical Validation](https://mlsanity.com/paper/2505.20340/) (2025)

## Built on by (most cited first, 10 of 10)

- [Before the Last Token: Diagnosing Final-Token Safety Probe Failures](https://mlsanity.com/paper/2605.12726/) (2026)
- [SliceGraph: Mapping Process Isomers in Multi-Run Chain-of-Thought Reasoning](https://mlsanity.com/paper/2605.14619/) (2026)
- [Reasoning Models Don't Just Think Longer, They Move Differently](https://mlsanity.com/paper/2605.15454/) (2026)
- [Entropy-Gradient Inversion: Moving Toward Internal Mechanism of Large Reasoning Models](https://mlsanity.com/paper/2605.17770/) (2026)
- [An Analysis of Residual-Stream Geometry Across Transformer Depth](https://mlsanity.com/paper/2607.18348/) (2026)
- [Geometry-Lite: Interpretable Safety Probing via Layer-Wise Margin Geometry](https://mlsanity.com/paper/2605.20241/) (2026)
- [D2-Monitor: Dynamic Safety Monitoring for Diffusion LLMs via Hesitation Signals](https://mlsanity.com/paper/2605.25893/) (2026)
- [Riemannian Geometry for Pre-trained Language Model Embeddings](https://mlsanity.com/paper/2607.07047/) (2026)
- [State of Thought Enables Endogenous Reasoning](https://mlsanity.com/paper/2609.16055/) (2026)
- [From Candidate Tokens to Final Outcomes: Decoding Relative Long-Horizon Candidate Value from Reasoning Model Representations](https://mlsanity.com/paper/2610.15412/) (2026)

## Abstract (arXiv)

Existing explainability methods for Large Language Models (LLMs) typically treat hidden states as static points in activation space, assuming that correct and incorrect inferences can be separated using representations from an individual layer. However, these activations are saturated with polysemantic features, leading to linear probes learning surface-level lexical patterns rather than underlying reasoning structures. We introduce Truth as a Trajectory (TaT), which models the transformer inference as an unfolded trajectory of iterative refinements, shifting analysis from static activations to layer-wise geometric displacement. By analyzing displacement of representations across layers, TaT uncovers geometric invariants that distinguish valid reasoning from spurious behavior. We evaluate TaT across dense and Mixture-of-Experts (MoE) architectures on benchmarks spanning commonsense reasoning, question answering, and toxicity detection. Without access to the activations themselves and using only changes in activations across layers, we show that TaT effectively mitigates reliance on static lexical confounds, outperforming conventional probing, and establishes trajectory analysis as a complementary perspective on LLM explainability.