mlsanityData to 7 Oct 2026

Reasoning as State Transition: A Representational Analysis of Reasoning Evolution in Large Language Models

arXiv 2602.00770 ↗First posted 31 Jan 2026cs.CLcited by 0

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Large Language Models have achieved remarkable performance on reasoning tasks, motivating research into how this ability evolves during training. Prior work has primarily analyzed this evolution via explicit generation outcomes, treating the reasoning process as a black box and obscuring internal changes. To address this opacity, we introduce a representational perspective to investigate the dynamics of the model's internal states. Through comprehensive experiments across models at various training stages, we discover that post-training yields only limited improvement in static initial representation quality. Furthermore, we reveal that, distinct from non-reasoning tasks, reasoning involves a significant continuous distributional shift in representations during generation. Comparative analysis indicates that post-training empowers models to drive this transition toward a better distribution for task solving. To clarify the relationship between internal states and external outputs, statistical analysis confirms a high correlation between generation correctness and the final representations; while counterfactual experiments identify the semantics of the generated tokens, rather than additional computation during inference or intrinsic parameter differences, as the dominant driver of the transition. Collectively, we offer a novel understanding of the reasoning process and the effect of training on reasoning enhancement, providing valuable insights for future model analysis and optimization.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 32 of its 72 references that are in the map. First posted on arXiv 31 Jan 2026.

Builds on

32 of 72 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025DeepDistill: Enhancing LLM Reasoning Capabilities via Large-Scale Difficulty-Graded Data Training24 Apr
  2. 2024Boosting Deductive Reasoning with Step Signals In RLHF12 Oct

Other references · 30

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2024GPT-4o System Cardcited by 4,467
  4. 2024Qwen2 Technical Reportcited by 2,076
  5. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,276
  6. 2024OpenAI o1 System Cardcited by 1,220
  7. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,193
  8. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,057
  9. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 948
  10. 2025Group Sequence Policy Optimizationcited by 788
  11. 2025Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoningcited by 540
  12. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wildcited by 505
18 more
  1. 2025Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRscited by 373
  2. 2025Spurious Rewards: Rethinking Training Signals in RLVRcited by 199
  3. 2024Let's Think Dot by Dot: Hidden Computation in Transformer Language Modelscited by 186
  4. 2024Teaching Large Language Models to Reason with Reinforcement Learningcited by 158
  5. 2025Reasoning Models Know When They're Right: Probing Hidden States for Self-Verificationcited by 154
  6. 2025LLMs Can Easily Learn to Reason from Demonstrations Structure, not content, is what matters!cited by 132
  7. 2025Echo Chamber: RL Post-training Amplifies Behaviors Learned in Pretrainingcited by 123
  8. 2025REASONING GYM: Reasoning Environments for Reinforcement Learning with Verifiable Rewardscited by 121
  9. 2025Reasoning with Sampling: Your Base Model is Smarter Than You Thinkcited by 84
  10. 2025On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Modelscited by 64
  11. 2025No Answer Needed: Predicting LLM Answer Accuracy from Question-Only Linear Probescited by 35
  12. 2025RL Squeezes, SFT Expands: A Comparative Study of Reasoning LLMscited by 32
  13. 2025Physics of Language Models: Part 4.1, Architecture Design and the Magic of Canon Layerscited by 32
  14. 2025CTRLS: Chain-of-Thought Reasoning via Latent State-Transitioncited by 13
  15. 2025CLUE: Non-parametric Verification from Experience via Hidden-State Clusteringcited by 12
  16. 2025Geometric Metrics and LLMs: What They Measure and When They Workcited by 4
  17. 2025Response Uncertainty and Probe Modeling: Two Sides of the Same Coin in LLM Interpretability?cited by 3
  18. 2025Explainability of Large Language Models: Opportunities and Challenges toward Generating Trustworthy Explanationscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Dissecting Failure Dynamics in Large Language Model Reasoning
  2. 2025Scaling Competence, Shrinking Reasoning: Cognitive Signatures in Language Model Learning
  3. 2026Convergence Without Understanding: When Language Models Agree on Representations but Disagree on Reasoning
  4. 2026Reasoning emerges from constrained inference manifolds in large language models
  5. 2026PRISM: A Dual View of LLM Reasoning through Semantic Flow and Latent Computation
  6. 2026Unveiling the Reasoning Process of Large Language Models
  7. 2026Spatiotemporal Hidden-State Dynamics as a Signature of Internal Reasoning in Large Language Models
  8. 2025State over Tokens: Characterizing the Role of Reasoning Tokens