mlsanityData to 14 Oct 2026

VeriThinker: Learning to Verify Makes Reasoning Model Efficient

arXiv 2505.17941 ↗First posted 23 May 2025cs.LGcited by 24

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Large Reasoning Models (LRMs) excel at complex tasks using Chain-of-Thought (CoT) reasoning. However, their tendency to overthinking leads to unnecessarily lengthy reasoning chains, dramatically increasing inference costs. To mitigate this issue, we introduce VeriThinker, a novel approach for CoT compression. Unlike conventional methods that fine-tune LRMs directly on the original reasoning task using synthetic concise CoT data, we innovatively fine-tune the model solely through an auxiliary verification task. By training LRMs to accurately verify the correctness of CoT solutions, the LRMs inherently become more discerning about the necessity of subsequent self-reflection steps, thereby effectively suppressing overthinking. Extensive experiments validate that VeriThinker substantially reduces reasoning chain lengths while maintaining or even slightly improving accuracy. When applied to DeepSeek-R1-Distill-Qwen-7B, our approach reduces reasoning tokens on MATH500 from 3790 to 2125 while improving accuracy by 0.8% (94.0% to 94.8%), and on AIME25, tokens decrease from 14321 to 10287 with a 2.1% accuracy gain (38.7% to 40.8%). Additionally, our experiments demonstrate that VeriThinker can also be zero-shot generalized to speculative reasoning. Code is available at https://github.com/czg1225/VeriThinker

Key facts. Cited by 24 papers in the map (data through 14 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 62 of its 73 references that are in the map. First posted on arXiv 23 May 2025.

Builds on

62 of 73 references.

Closest ancestors · 25

Same direction, newest first

  1. 2025Thinkless: LLM Learns When to Think19 May
  2. 2025Dynamic Early Exit in Reasoning Models22 Apr
  3. 2025Learning Adaptive Parallel Reasoning with Language Models21 Apr
  4. 2025An Empirical Study of LLM Reasoning Ability Under Strict Output Length Constraint19 Apr
  5. 2025Rethinking Inference-Time Scaling: Efficiency Limits and Linguistic Signals18 Apr
  6. 2025Thought Manipulation: External Thought Can Be Efficient for Large Reasoning Models18 Apr
  7. 2025SpecReason: Fast and Accurate Inference-Time Compute via Speculative Reasoning10 Apr
  8. 2025Quantization Hurts Reasoning? An Empirical Study on Quantized Reasoning Models7 Apr
  9. 2025When Reasoning Meets Compression: Understanding the Effects of LLMs Compression on Large Reasoning Models2 Apr
  10. 2025ThinkPrune: Pruning Long Chain-of-Thought of LLMs via Reinforcement Learning2 Apr
  11. 2025L1: Controlling How Long A Reasoning Model Thinks With Reinforcement Learning6 Mar
  12. 2025How Well do LLMs Compress Their Own Chain-of-Thought? A Token Complexity Approach3 Mar
13 more
  1. 2025Self-Training Elicits Concise Reasoning in Large Language Models27 Feb
  2. 2025Towards Thinking-Optimal Scaling of Test-Time Compute for LLM Reasoning25 Feb
  3. 2025TokenSkip: Controllable Chain-of-Thought Compression in LLMs17 Feb
  4. 2025CoT-Valve: Length-Compressible Chain-of-Thought Tuning13 Feb
  5. 2025Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach7 Feb
  6. 2025Demystifying Long Chain-of-Thought Reasoning in LLMs5 Feb
  7. 2025Reward-Guided Speculative Decoding for Efficient LLM Reasoning31 Jan
  8. 2025Efficient Reasoning with Hidden Thinking31 Jan
  9. 2025O1-Pruner: Length-Harmonizing Fine-Tuning for O1-Like Reasoning Pruning22 Jan
  10. 2024Do NOT Think That Much for 2+3=? On the Overthinking of o1-Like LLMs30 Dec
  11. 2024Token-Budget-Aware LLM Reasoning24 Dec
  12. 2024EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty26 Jan
  13. 2023Skeleton-of-Thought: Prompting LLMs for Efficient Parallel Generation28 Jul

Other references · 37

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  3. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  4. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  5. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  6. 2024OpenAI o1 System Cardcited by 1,244
  7. 2025Kimi k1.5: Scaling Reinforcement Learning with LLMscited by 1,210
  8. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
  9. 2024Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Headscited by 843
  10. 2024Training Large Language Models to Reason in a Continuous Latent Spacecited by 797
  11. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 665
  12. 2025LIMO: Less is More for Reasoningcited by 498
25 more
  1. 2024LLaVA-CoT: Let Vision Language Models Reason Step-by-Stepcited by 454
  2. 2025rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinkingcited by 313
  3. 2025Chain of Draft: Thinking Faster by Writing Lesscited by 297
  4. 2024Unlocking Efficiency in Large Language Model Inference: A Comprehensive Survey of Speculative Decodingcited by 269
  5. 2025Training Language Models to Reason Efficientlycited by 247
  6. 2024From Explicit CoT to Implicit CoT: Learning to Internalize CoT Step by Stepcited by 235
  7. 2025Inference-Time Scaling for Diffusion Models beyond Scaling Denoising Stepscited by 234
  8. 2025Imagine while Reasoning in Space: Multimodal Visualization-of-Thoughtcited by 206
  9. 2023Implicit Chain of Thought Reasoning via Knowledge Distillationcited by 176
  10. 2025T2I-R1: Reinforcing Image Generation with Collaborative Semantic-level and Token-level CoTcited by 166
  11. 2025SANA 1.5: Efficient Scaling of Training-Time and Inference-Time Compute in Linear Diffusion Transformercited by 165
  12. 2024Fast Best-of-N Decoding via Speculative Rejectioncited by 112
  13. 2025One-Minute Video Generation with Test-Time Trainingcited by 101
  14. 2024Can Language Models Learn to Skip Steps?cited by 59
  15. 2025Phi-4-Mini-Reasoning: Exploring the Limits of Small Reasoning Language Models in Mathcited by 44
  16. 2025Unveiling the Key Factors for Distilling Chain-of-Thought Reasoningcited by 40
  17. 2025Video-T1: Test-Time Scaling for Video Generationcited by 40
  18. 2025Think When You Need: Self-Adaptive Chain-of-Thought Learningcited by 38
  19. 2025Ada-R1: Hybrid-CoT via Bi-Level Adaptive Reasoning Optimizationcited by 33
  20. 2025Towards Reasoning Ability of Small Language Modelscited by 29
  21. 2024Non-myopic Generation of Language Models for Reasoning and Planningcited by 26
  22. 2025Introducing Visual Perception Token into Multimodal Large Language Modelcited by 26
  23. 2025Adaptive Group Policy Optimization: Towards Stable Training and Token-Efficient Reasoningcited by 21
  24. 2023Mixed Distillation Helps Smaller Language Model Better Reasoningcited by 19
  25. 2024Improving Mathematical Reasoning Capabilities of Small Language Models via Feedback-Driven Distillationcited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 12; 2026: 12.

24 papers.

Sort:

By area:

  1. 2025Don't Overthink It: A Survey of Efficient R1-style Large Reasoning Modelscited by 37
  2. 2025OptimalThinkingBench: Evaluating Over and Underthinking in LLMscited by 35
  3. 2025DLER: Doing Length pEnalty Right - Incentivizing More Intelligence per Token via Reinforcement Learningcited by 35
  4. 2026Render-of-Thought: Rendering Textual Chain-of-Thought as Images for Visual Latent Reasoningcited by 20
  5. 2025OBS-Diff: Accurate Pruning For Diffusion Models in One-Shotcited by 19
  6. 2025Towards Concise and Adaptive Thinking in Large Reasoning Models: A Surveycited by 18
  7. 2025Performative Thinking? The Brittle Correlation Between CoT Length and Problem Complexitycited by 9
  8. 2026Neural Chain-of-Thought Search: Searching the Optimal Reasoning Path to Enhance Large Language Modelscited by 8
  9. 2026ORBIT: On-policy Exploration-Exploitation for Controllable Multi-Budget Reasoningcited by 7
  10. 2025DTS: Enhancing Large Reasoning Models via Decoding Tree Sketchingcited by 6
  11. 2025Thinking-Free Policy Initialization Makes Distilled Reasoning Models More Effective and Efficient Reasonerscited by 5
  12. 2025MixReasoning: Switching Modes to Thinkcited by 3
12 more
  1. 2025Entropy-Guided Reasoning Compressioncited by 3
  2. 2026BFS-PO: Best-First Search for Large Reasoning Modelscited by 2
  3. 2026Learning at the Right Pace: Adaptive Data Scheduling Improves LLM Reinforcement Learningcited by 2
  4. 2026SketchThinker-R1: Towards Efficient Sketch-Style Reasoning in Large Multimodal Modelscited by 1
  5. 2026A State-Transition Framework for Efficient LLM Reasoningcited by 1
  6. 2026Hint Tuning: Less Data Makes Better Reasonerscited by 1
  7. 2026When Implausible Tokens Get Reinforced: Tail-Aware Credit Calibration for LLM Reinforcement Learningcited by 1
  8. 2025Rewarding Structural Conformance of Reasoning using Process Miningcited by 0
  9. 2025Hard Negative Sample-Augmented DPO Post-Training for Small Language Modelscited by 0
  10. 2026Shorter, but Still Trustworthy? An Empirical Study of Chain-of-Thought Compressioncited by 0
  11. 2026Graph-Based Chain-of-Thought Pruning for Reducing Redundant Reflections in Reasoning LLMscited by 0
  12. 2026Nice Fold or Hero Call: Learning Budget-Efficient Thinking for Adaptive Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025TrimR: Verifier-based Training-Free Thinking Compression for Efficient Test-Time Scaling
  2. 2026HybridThinker: Efficient Chain-of-Thought Reasoning via Compressed Memory and Transient Thought Steps
  3. 2026Towards Efficient Large Language Reasoning Models via Extreme-Ratio Chain-of-Thought Compression
  4. 2026TERMINATOR: Learning Optimal Exit Points for Early Stopping in Chain-of-Thought Reasoning
  5. 2025Efficient Inference for Large Reasoning Models: A Survey
  6. 2025A Survey of Efficient Reasoning for Large Reasoning Models: Language, Multimodality, and Beyond
  7. 2026Think Less, Know More: State-Aware Reasoning Compression with Knowledge Guidance for Efficient Reasoning
  8. 2025Efficient Reasoning Models: A Survey