mlsanityData to 14 Oct 2026

From Solving to Verifying: A Unified Objective for Robust Reasoning in LLMs

arXiv 2511.15137 ↗First posted 19 Nov 2025cs.LG cs.AIcited by 6

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

The reasoning capabilities of large language models (LLMs) have been significantly improved through reinforcement learning (RL). Nevertheless, LLMs still struggle to consistently verify their own reasoning traces. This raises the research question of how to enhance the self-verification ability of LLMs and whether such an ability can further improve reasoning performance. In this work, we propose GRPO-Verif, an algorithm that jointly optimizes solution generation and self-verification within a unified loss function, with an adjustable hyperparameter controlling the weight of the verification signal. Experimental results demonstrate that our method enhances self-verification capability while maintaining comparable performance in reasoning.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 28 of its 30 references that are in the map. First posted on arXiv 19 Nov 2025.

Builds on

28 of 30 references.

Closest ancestors · 7

Same direction, newest first

  1. 2025Critique-GRPO: Advancing LLM Reasoning with Natural Language and Numerical Feedback3 Jun
  2. 2025RM-R1: Reward Modeling as Reasoning5 May
  3. 2025Inference-Time Scaling for Generalist Reward Modeling3 Apr
  4. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  5. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning22 Jan
  6. 2024Generative Verifiers: Reward Modeling as Next-Token Prediction27 Aug
  7. 2023Let's Verify Step by Step31 May

Other references · 21

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  3. 2024Qwen2.5 Technical Reportcited by 4,942
  4. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  5. 2024Qwen2 Technical Reportcited by 2,120
  6. 2022Solving Quantitative Reasoning Problems with Language Modelscited by 1,392
  7. 2024OlympiadBench: A Challenging Benchmark for Promoting AGI with Olympiad-Level Bilingual Multimodal Scientific Problemscited by 1,136
  8. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 1,017
  9. 2023Teaching Large Language Models to Self-Debugcited by 988
  10. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 940
  11. 2022Galactica: A Large Language Model for Sciencecited by 744
  12. 2024Training Language Models to Self-Correct via Reinforcement Learningcited by 426
9 more
  1. 2023Beyond Human Data: Scaling Self-Training for Problem-Solving with Language Modelscited by 349
  2. 2023SciBench: Evaluating College-Level Scientific Problem-Solving Abilities of Large Language Modelscited by 256
  3. 2024Generative Reward Modelscited by 148
  4. 2025Self-rewarding correction for mathematical reasoningcited by 70
  5. 2025Trust, But Verify: A Self-Verification Approach to Reinforcement Learning with Verifiable Rewardscited by 38
  6. 2025PAG: Multi-Turn Reinforced LLM Self-Correction with Policy as Generative Verifiercited by 28
  7. 2025Heimdall: test-time scaling on the generative verificationcited by 26
  8. 2025A Unified Pairwise Framework for RLHF: Bridging Generative Reward Modeling and Policy Optimizationcited by 21
  9. 2025Entropy-Based Adaptive Weighting for Self-Trainingcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 6.

6 papers.

  1. 2026Learning to Self-Verify Makes Language Models Better Reasonerscited by 21
  2. 2026Blockwise Advantage Estimation for Multi-Objective RL with Verifiable Rewardscited by 4
  3. 2026Pseudo-Formalization for Automatic Proof Verificationcited by 4
  4. 2026Towards Better RL Training Data Utilization via Second-Order Rolloutcited by 1
  5. 2026SLCA-GRPO: Resolving Cross-Segment Credit Misattribution in Tool-Calling RLcited by 1
  6. 2026Why Deterministic PRM Guidance Underperforms in Discrete Diffusion Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Incentivizing LLMs to Self-Verify Their Answers
  2. 2026Reinforcing Step-level Reasoning for Effective Self-Correction in LLMs
  3. 2025TinyV: Reducing False Negatives in Verification Improves RL for LLM Reasoning
  4. 2026Re2: Unlocking LLM Reasoning via Reinforcement Learning with Re-solving
  5. 2025VerifyBench: A Systematic Benchmark for Evaluating Reasoning Verifiers Across Domains
  6. 2025How Difficulty-Aware Staged Reinforcement Learning Enhances LLMs' Reasoning Capabilities: A Preliminary Experimental Study
  7. 2025Reinforcing General Reasoning without Verifiers
  8. 2024Learning From Correctness Without Prompting Makes LLM Efficient Reasoner