Spatiotemporal Hidden-State Dynamics as a Signature of Internal Reasoning in Large Language Models
On the map: LLM reasoning → Reasoning benchmarks and evaluation
Abstract (arXiv)
Large reasoning models (LRMs) generate extended solutions, yet it remains unclear whether these traces reflect substantive internal computation or merely verbosity and overthinking. Although recent hidden-state analyses suggest that internal representations carry correctness-related signals, their coarse aggregations may obscure the token and layer structure underlying reasoning computation. We investigate hidden-state transitions across decoding steps and layers, and identify a distinct spatiotemporal pattern in LRMs: successful trajectories exhibit broad temporal dynamics with localized layer-wise concentration, while this structure is weaker in non-reasoning models and knowledge-heavy domains. We formalize this characteristic as Spatiotemporal Amplitude of Latent Transition (StALT), a training-free trajectory statistic that summarizes temporal changes between adjacent tokens weighted by within-token layer saliency. Across diverse models and benchmarks, StALT reliably separates correct from incorrect trajectories in reasoning-intensive regimes, providing a competitive label-free correctness signal alongside strong output-space and length-based baselines. Intervention analyses further show that this spatiotemporal amplitude responds systematically to manipulations that increase or reduce the demand for internal reasoning, supporting its association with latent reasoning dynamics in LRMs. These findings provide empirical evidence that LRMs exhibit measurable hidden-state dynamics and offer a practical probe for understanding internal computation beyond output-based evaluation.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 22 of its 45 references that are in the map. First posted on arXiv 3 May 2026.
Builds on
22 of 45 references.
Closest ancestors · 1
Same direction, newest first
- 2025Qwen3 Technical Report14 May
Other references · 21
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2021Training Verifiers to Solve Math Word Problemscited by 9,743
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,258
- 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,057
- 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 946
- 2022Teaching Models to Express Their Uncertainty in Wordscited by 731
- 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 695
- 2024TrustLLM: Trustworthiness in Large Language Modelscited by 280
- 2025When More is Less: Understanding Chain-of-Thought Length in LLMscited by 182
- 2025Reasoning Models Know When They're Right: Probing Hidden States for Self-Verificationcited by 154
9 more
- 2025Confidence Is All You Need: Few-Shot RL Fine-Tuning of Language Modelscited by 61
- 2025CoT-Kinetics: A Theoretical Modeling Assessing LRM Reasoning Processcited by 50
- 2025Overclocking LLM Reasoning: Monitoring and Controlling Thinking Path Lengths in LLMscited by 22
- 2026Think Deep, Not Just Long: Measuring LLM Reasoning Effort via Deep-Thinking Tokenscited by 21
- 2025Knowledge or Reasoning? A Close Look at How LLMs Think Across Domainscited by 14
- 2025CLUE: Non-parametric Verification from Experience via Hidden-State Clusteringcited by 12
- 2025Can LLMs Predict Their Own Failures? Self-Awareness via Internal Circuitscited by 12
- 2026Hidden States as Early Signals: Step-level Trace Evaluation and Pruning for Efficient Test-Time Scalingcited by 10
- 2025Tracing the Traces: Latent Temporal Signals for Efficient and Accurate Reasoningcited by 8
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Reasoning as State Transition: A Representational Analysis of Reasoning Evolution in Large Language Models
- 2026Reasoning emerges from constrained inference manifolds in large language models
- 2026Unveiling the Reasoning Process of Large Language Models
- 2026Dissecting Failure Dynamics in Large Language Model Reasoning
- 2026The Tell-Tale Norm: ℓ2 Magnitude as a Signal for Reasoning Dynamics in Large Language Models
- 2026Truth as a Trajectory: What Internal Representations Reveal About Large Language Model Reasoning
- 2025A Statistical Physics of Language Model Reasoning
- 2026LLM Reasoning Is Latent, Not the Chain of Thought