mlsanityData to 14 Oct 2026

Reasoning Models Know When They're Right: Probing Hidden States for Self-Verification

arXiv 2504.05419 ↗First posted 7 Apr 2025cs.AI cs.CLcited by 159

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Reasoning models have achieved remarkable performance on tasks like math and logical reasoning thanks to their ability to search during reasoning. However, they still suffer from overthinking, often performing unnecessary reasoning steps even after reaching the correct answer. This raises the question: can models evaluate the correctness of their intermediate answers during reasoning? In this work, we study whether reasoning models encode information about answer correctness through probing the model's hidden states. The resulting probe can verify intermediate answers with high accuracy and produces highly calibrated scores. Additionally, we find models' hidden states encode correctness of future answers, enabling early prediction of the correctness before the intermediate answer is fully formulated. We then use the probe as a verifier to decide whether to exit reasoning at intermediate answers during inference, reducing the number of inference tokens by 24% without compromising performance. These findings confirm that reasoning models do encode a notion of correctness yet fail to exploit it, revealing substantial untapped potential to enhance their efficiency.

Key facts. Cited by 159 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 33 of its 42 references that are in the map. First posted on arXiv 7 Apr 2025.

Builds on

33 of 42 references.

Closest ancestors · 11

Same direction, newest first

  1. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models20 Mar
  2. 2025Self-Training Elicits Concise Reasoning in Large Language Models27 Feb
  3. 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs30 Dec
  4. 2024Efficiently Scaling LLM Reasoning with Certaindex30 Dec
  5. 2024Learning How Hard to Think: Input-Adaptive Allocation of LM Computation7 Oct
  6. 2024Adaptive Inference-Time Compute: LLMs Can Predict if They Can Do Better, Even Mid-Generation3 Oct
  7. 2024Make Every Penny Count: Difficulty-Adaptive Self-Consistency for Cost-Efficient Reasoning24 Aug
  8. 2024Small Language Models Need Strong Verifiers to Self-Correct Reasoning26 Apr
  9. 2024Escape Sky-high Cost: Early-stopping Self-Consistency for Multi-step Reasoning19 Jan
  10. 2023REFINER: Reasoning Feedback on Intermediate Representations4 Apr
  11. 2023Self-Refine: Iterative Refinement with Self-Feedback30 Mar

Other references · 22

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  5. 2024Qwen2.5 Technical Reportcited by 4,942
  6. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  7. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 974
  8. 2021Probing Classifiers: Promises, Shortcomings, and Advancescited by 876
  9. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 812
  10. 2022Discovering Latent Knowledge in Language Models Without Supervisioncited by 767
  11. 2022Teaching Models to Express Their Uncertainty in Wordscited by 751
  12. 2024Generative Verifiers: Reward Modeling as Next-Token Predictioncited by 480
10 more
  1. 2022Large Language Models are Better Reasoners with Self-Verificationcited by 307
  2. 2020Reducing conversational agents' overconfidence through linguistic calibrationcited by 210
  3. 2023Solving Challenging Math Word Problems Using GPT-4 Code Interpreter with Code-based Self-Verificationcited by 166
  4. 2022Faithful Reasoning Using Large Language Modelscited by 130
  5. 2023Deductive Verification of Chain-of-Thought Reasoningcited by 124
  6. 2024Large Language Models Must Be Taught to Know What They Don't Knowcited by 100
  7. 2025Logical Reasoning in Large Language Models: A Surveycited by 55
  8. 2023Language Models can be Logical Solverscited by 24
  9. 2024Automatic Curriculum Expert Iteration for Reliable LLM Reasoningcited by 16
  10. 2025SCoRE: Benchmarking Long-Chain Reasoning in Commonsense Scenarioscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 56; 2026: 103.

159 papers.

Sort:

By area:

  1. 2025Dynamic Early Exit in Reasoning Modelscited by 219
  2. 2025Wait, We Don't Need to "Wait"! Removing Thinking Tokens Improves Reasoning Efficiencycited by 66
  3. 2025Reasoning Models Better Express Their Confidencecited by 65
  4. 2025DeepAnalyze: Agentic Large Language Models for Autonomous Data Sciencecited by 50
  5. 2026Reasoning Theater: Disentangling Model Beliefs from Chain-of-Thoughtcited by 40
  6. 2025Don't Overthink It: A Survey of Efficient R1-style Large Reasoning Modelscited by 37
  7. 2025FlashThink: An Early Exit Method For Efficient Reasoningcited by 28
  8. 2025Let LRMs Break Free from Overthinking via Self-Braking Tuningcited by 27
  9. 2025When to Continue Thinking: Adaptive Thinking Mode Switching for Efficient Reasoningcited by 26
  10. 2025A Comprehensive Survey on Trustworthiness in Reasoning with Large Language Modelscited by 25
  11. 2025What Characterizes Effective Reasoning? Revisiting Length, Review, and Structure of CoTcited by 25
  12. 2025Entropy After </Think> for reasoning model early exitingcited by 24
38 more
  1. 2026LLM Reasoning as Trajectories: Step-Specific Representation Geometry and Correctness Signalscited by 24
  2. 2025Thought calibration: Efficient and confident test-time scalingcited by 21
  3. 2025Trace Length is a Simple Uncertainty Signal in Reasoning Modelscited by 19
  4. 2025The Price of a Second Thought: On the Evaluation of Reasoning Efficiency in Large Language Modelscited by 18
  5. 2025Towards Concise and Adaptive Thinking in Large Reasoning Models: A Surveycited by 18
  6. 2025Can We Predict Alignment Before Models Finish Thinking? Towards Monitoring Misaligned Reasoning Modelscited by 17
  7. 2025Can LLMs Predict Their Own Failures? Self-Awareness via Internal Circuitscited by 14
  8. 2026Parallel-Probe: Towards Efficient Parallel Thinking via 2D Probingcited by 14
  9. 2026NeuReasoner: Towards Explainable, Controllable, and Unified Reasoning via Mixture-of-Neuronscited by 14
  10. 2025The Geometry of Self-Verification in a Task-Specific Reasoning Modelcited by 13
  11. 2026Real-Time Aligned Reward Model beyond Semanticscited by 13
  12. 2025Why Do Multilingual Reasoning Gaps Emerge in Reasoning Language Models?cited by 12
  13. 2025Unsupervised Elicitation of Language Modelscited by 11
  14. 2025The First Impression Problem: Internal Bias Triggers Overthinking in Reasoning Modelscited by 10
  15. 2025Large Language Models Have Intrinsic Meta-Cognition, but Need a Good Lenscited by 10
  16. 2026Conformal Thinking: Risk Control for Reasoning on a Compute Budgetcited by 10
  17. 2026LLMs Improving LLMs: Agentic Discovery for Test-Time Scalingcited by 10
  18. 2026When Agents Commit Too Soon: Diagnosing Premature Commitment in LLM Agentscited by 10
  19. 2025Mining Intrinsic Rewards from LLM Hidden States for Efficient Best-of-N Samplingcited by 9
  20. 2025The Geometries of Truth Are Orthogonal Across Taskscited by 9
  21. 2025SpecExit: Accelerating Large Reasoning Model via Speculative Exitcited by 9
  22. 2025Tracing the Traces: Latent Temporal Signals for Efficient and Accurate Reasoningcited by 9
  23. 2025LLM Knowledge is Brittle: Truthfulness Representations Rely on Superficial Resemblancecited by 9
  24. 2025Calibrating LLM Judges: Linear Probes for Fast and Reliable Uncertainty Estimationcited by 9
  25. 2025Exploring and Exploiting the Inherent Efficiency within Large Reasoning Models for Self-Guided Efficiency Enhancementcited by 8
  26. 2026Lie to Me: How Faithful Is Chain-of-Thought Reasoning in Reasoning Models?cited by 8
  27. 2026On Calibration of Large Language Models: From Response To Capabilitycited by 7
  28. 2026Hidden Error Awareness in Chain-of-Thought Reasoning: The Signal Is Diagnostic, Not Causalcited by 7
  29. 2026The Value Axis: Language Models Encode Whether They're on the Right Trackcited by 7
  30. 2025When Do LLMs Admit Their Mistakes? Understanding The Role Of Model Belief In Retractioncited by 6
  31. 2025SCI-Verifier: Scientific Verifier with Thinkingcited by 6
  32. 2025Hallucination reduction with CASAL: Contrastive Activation Steering For Amortized Learningcited by 6
  33. 2025Internal states before wait modulate reasoning patternscited by 6
  34. 2025LYNX: Learning Dynamic Exits for Confidence-Controlled Reasoningcited by 6
  35. 2026Towards a Mechanistic Understanding of Large Reasoning Models: A Survey of Training, Inference, and Failurescited by 6
  36. 2026Statistical Early Stopping for Reasoning Modelscited by 6
  37. 2026Post-Hoc Reasoning in Chain of Thought: Decoding and Steering Pre-Committed Answerscited by 6
  38. 2026Mitigating Overthinking in Large Reasoning Language Models via Reasoning Path Deviation Monitoringcited by 6

50 most cited of 159 All 159 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Catch Me If You Can: How Smaller Reasoning Models Pretend to Reason with Mathematical Fidelity
  2. 2026Reasoning-Trace Collapse: Evaluating the Loss of Explicit Reasoning During Fine-Tuning
  3. 2026Does Inference Scaling Improve Reasoning Faithfulness? A Multi-Model Analysis of Self-Consistency Tradeoffs
  4. 2025The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity
  5. 2026Stop When Reasoning Converges: Semantic-Preserving Early Exit for Reasoning Models
  6. 2026A Reference-Free Score for Detecting Silent Reasoning Failures in Large Language Models
  7. 2025First Try Matters: Revisiting the Role of Reflection in Reasoning Models
  8. 2025Answering the Unanswerable Is to Err Knowingly: Analyzing and Mitigating Abstention Failures in Large Reasoning Models