mlsanityData to 7 Oct 2026

Learning to Self-Verify Makes Language Models Better Reasoners

arXiv 2602.07594 ↗First posted 7 Feb 2026cs.CL cs.AIcited by 18

On the map: LLM reasoning → Test-time compute and efficient thinking

Abstract (arXiv)

Recent large language models (LLMs) achieve strong performance in generating promising reasoning paths for complex tasks. However, despite powerful generation ability, LLMs remain weak at verifying their own answers, revealing a persistent capability asymmetry between generation and self-verification. In this work, we conduct an in-depth investigation of this asymmetry throughout training evolution and show that, even on the same task, improving generation does not lead to corresponding improvements in self-verification. Interestingly, we find that the reverse direction of this asymmetry behaves differently: learning to self-verify can effectively improve generation performance, achieving accuracy comparable to standard generation training while yielding more efficient and effective reasoning traces. Building on this observation, we further explore integrating self-verification into generation training by formulating a multi-task reinforcement learning framework, where generation and self-verification are optimized as two independent but complementary objectives. Extensive experiments across benchmarks and models demonstrate performance gains over generation-only training in both generation and verification capabilities.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Placed in Test-time compute and efficient thinking, one of 17 directions in LLM reasoning. Builds on 47 of its 53 references that are in the map. First posted on arXiv 7 Feb 2026.

Builds on

47 of 53 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025Incentivizing LLMs to Self-Verify Their Answers2 Jun
  2. 2025Efficient Reasoning Models: A Survey15 Apr
  3. 2025Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Models31 Mar
  4. 2025Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models20 Mar
  5. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters6 Aug
  6. 2024Small Language Models Need Strong Verifiers to Self-Correct Reasoning26 Apr
  7. 2024On the Self-Verification Limitations of Large Language Models on Reasoning and Planning Tasks12 Feb

Other references · 40

  1. 2020Language Models are Few-Shot Learnerscited by 32,199
  2. 2023GPT-4 Technical Reportcited by 18,132
  3. 2022Training language models to follow instructions with human feedbackcited by 15,712
  4. 2017Proximal Policy Optimization Algorithmscited by 14,441
  5. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  6. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  7. 2025Qwen3 Technical Reportcited by 8,195
  8. 2023Let's Verify Step by Stepcited by 3,598
  9. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  10. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  11. 2025Gemma 3 Technical Reportcited by 1,686
  12. 2025Search-R1: Training LLMs to Reason and Leverage Search Engines with Reinforcement Learningcited by 1,371
28 more
  1. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,099
  2. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wildcited by 505
  3. 2024Generative Verifiers: Reward Modeling as Next-Token Predictioncited by 465
  4. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Modelcited by 450
  5. 2025RAGEN: Understanding Self-Evolution in LLM Agents via Multi-Turn Reinforcement Learningcited by 357
  6. 2025Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domainscited by 328
  7. 2024Skywork-Reward: Bag of Tricks for Reward Modeling in LLMscited by 322
  8. 2025Inference-Time Scaling for Generalist Reward Modelingcited by 233
  9. 2025Skywork-Reward-V2: Scaling Preference Data Curation via Human-AI Synergycited by 204
  10. 2025Crossing the Reward Bridge: Expanding RL with Verifiable Rewards Across Diverse Domainscited by 150
  11. 2025RM-R1: Reward Modeling as Reasoningcited by 149
  12. 2024Generative Reward Modelscited by 148
  13. 2025Pass@k Training for Adaptively Balancing Exploration and Exploitation of Large Reasoning Modelscited by 134
  14. 2025DeepSeekMath-V2: Towards Self-Verifiable Mathematical Reasoningcited by 84
  15. 2025RLPR: Extrapolating RLVR to General Domains without Verifierscited by 83
  16. 2024Bootstrapping Language Models with DPO Implicit Rewardscited by 53
  17. 2025A Comprehensive Survey of Reward Models: Taxonomy, Applications, Challenges, and Futurecited by 53
  18. 2023A Closer Look at the Self-Verification Abilities of Large Language Models in Logical Reasoningcited by 41
  19. 2025Depth-Breadth Synergy in RLVR: Unlocking LLM Reasoning Gains with Adaptive Explorationcited by 40
  20. 2025Trust, But Verify: A Self-Verification Approach to Reinforcement Learning with Verifiable Rewardscited by 38
  21. 2025RewardAnything: Generalizable Principle-Following Reward Modelscited by 27
  22. 2025Reward Models in Deep Reinforcement Learning: A Surveycited by 24
  23. 2024Dissociation of Faithful and Unfaithful Reasoning in LLMscited by 23
  24. 2025Language Models that Think, Chat Bettercited by 23
  25. 2025Can Aha Moments Be Fake? Towards Quantifying Decorative and True Thinking in Chain-of-Thoughtcited by 14
  26. 2025When Does Verification Pay Off? A Closer Look at LLMs as Solution Verifierscited by 11
  27. 2025From Solving to Verifying: A Unified Objective for Robust Reasoning in LLMscited by 6
  28. 2025Zero Reinforcement Learning Towards General Domainscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 18.

18 papers.

Sort:

By area:

  1. 2026SKILL0: In-Context Agentic Reinforcement Learning for Skill Internalizationcited by 82
  2. 2026Self-Distilled Agentic Reinforcement Learningcited by 49
  3. 2026AgentOPSD: Recursive Self-Distillation for Agentic Reinforcement Learningcited by 8
  4. 2026Internalizing Safety Understanding in Large Reasoning Models via Verificationcited by 3
  5. 2026ICRL: Learning to Internalize Self-Critique with Reinforcement Learningcited by 3
  6. 2026SkillRise: Agentic Reinforcement Learning for Cross-Task Skill Evolutioncited by 3
  7. 2026Preference-Aware Rubric Learning for Personalized Evaluationcited by 2
  8. 2026CAST: Game Solvers as Turn-Level Teachers for LLM Agentscited by 2
  9. 2026Scaling Large Reasoning Models beyond Human Supervision: A Path toward Superintelligencecited by 2
  10. 2026MAP: A Map-then-Act Paradigm for Long-Horizon Interactive Agent Reasoningcited by 1
  11. 2026Exploiting Verification-Generation Gap: Test-Time Reinforcement Learning with Confidence-Conditioned Verificationcited by 1
  12. 2026Metacognition in LLMs: Foundations, Progress, and Opportunitiescited by 1
6 more
  1. 2026Self-Correction Can Amplify Hallucinations: Fact-Level Repair with Graph-Based Evidence Routing in Multimodal Generationcited by 0
  2. 2026APTER: Adaptive Post-Training with Expert-Grounded Rubricscited by 0
  3. 2026Coalition-Aware Skill Reliability for Self-Evolving Agentscited by 0
  4. 2026When Do Models Admit They Are Wrong? Failure Disclosure Is Unstable Under Reinforcement Learningcited by 0
  5. 2026Can Terminal Agents Trust Their Own Verification? Diagnosing and Improving Self-Verificationcited by 0
  6. 2026Finding the Right Fit: Model-Harness Interactions across Agent Taskscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Mind the Gap: Examining the Self-Improvement Capabilities of Large Language Models
  2. 2026On the Generalization Gap in Self-Evolving Language Model Reasoning
  3. 2026Training Large Language Models for Self-Explanation Faithfulness
  4. 2025PAG: Multi-Turn Reinforced LLM Self-Correction with Policy as Generative Verifier
  5. 2025The Path of Self-Evolving Large Language Models: Achieving Data-Efficient Learning via Intrinsic Feedback
  6. 2026Mid-Training with Self-Generated Data Improves Reinforcement Learning in Language Models
  7. 2025Self-Reflective Generation at Test Time
  8. 2025ReVISE: Learning to Refine at Test-Time via Intrinsic Self-Verification