mlsanityData to 14 Oct 2026

Decoupling Reasoning and Confidence: Resurrecting Calibration in Reinforcement Learning from Verifiable Rewards

arXiv 2603.09117 ↗First posted 10 Mar 2026cs.LG cs.AI cs.CLcited by 14

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Reinforcement Learning from Verifiable Rewards (RLVR) significantly enhances large language models (LLMs) reasoning but severely suffers from calibration degeneration, where models become excessively over-confident in incorrect answers. Previous studies devote to directly incorporating calibration objective into existing optimization target. However, our theoretical analysis demonstrates that there exists a fundamental gradient conflict between the optimization for maximizing policy accuracy and minimizing calibration error. Building on this insight, we propose DCPO, a simple yet effective framework that systematically decouples reasoning and calibration objectives. Extensive experiments demonstrate that our DCPO not only preserves accuracy on par with GRPO but also achieves the best calibration performance and substantially mitigates the over-confidence issue. Our study provides valuable insights and practical solution for more reliable LLM deployment.

Key facts. Cited by 14 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 29 of its 48 references that are in the map. First posted on arXiv 10 Mar 2026.

Builds on

29 of 48 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025C2GSPG: Confidence-calibrated Group Sequence Policy Gradient towards Self-aware Reasoning27 Sep
  2. 2025Uncalibrated Reasoning: GRPO Induces Overconfidence for Stochastic Outcomes15 Aug
  3. 2025Beyond Binary Rewards: Training LMs to Reason About Their Uncertainty22 Jul
  4. 2025Confidence Is All You Need: Few-Shot RL Fine-Tuning of Language Models5 Jun
  5. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model31 Mar
  6. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Training22 Nov

Other references · 23

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,371
  4. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  5. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  6. 2024OpenAI o1 System Cardcited by 1,244
  7. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
  8. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 974
  9. 2025Group Sequence Policy Optimizationcited by 835
  10. 2022Teaching Models to Express Their Uncertainty in Wordscited by 751
  11. 2023Generating with Confidence: Uncertainty Quantification for Black-box Large Language Modelscited by 293
  12. 2025AbstentionBench: Reasoning LLMs Fail on Unanswerable Questionscited by 145
11 more
  1. 2024Taming Overconfidence in LLMs: Reward Calibration in RLHFcited by 112
  2. 2024SaySelf: Teaching LLMs to Express Confidence with Self-Reflective Rationalescited by 105
  3. 2024On Verbalized Confidence Scores for LLMscited by 83
  4. 2025Reasoning Models Better Express Their Confidencecited by 65
  5. 2025Are Reasoning Models More Prone to Hallucination?cited by 64
  6. 2025Mind the Confidence Gap: Overconfidence, Calibration, and Distractor Effects in Large Language Modelscited by 40
  7. 2024Rowen: Adaptive Retrieval-Augmented Generation for Hallucination Mitigation in LLMscited by 27
  8. 2025Towards Fully Exploiting LLM Internal States to Enhance Knowledge Boundary Perceptioncited by 27
  9. 2024Do LLMs estimate uncertainty well in instruction-following?cited by 26
  10. 2025SConU: Selective Conformal Uncertainty in Large Language Modelscited by 23
  11. 2025Towards Harmonized Uncertainty Estimation for Large Language Modelscited by 3

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 14.

14 papers.

Sort:

By area:

  1. 2026Process Supervision of Confidence Margin for Calibrated LLM Reasoningcited by 2
  2. 2026Verifiable Rewards for Calibrated Probabilistic Forecastingcited by 2
  3. 2026What LLM Forecasters Know but Don't Say: Probing Internal Representations for Calibration and Faithfulnesscited by 2
  4. 2026CALIBER: Calibrating Confidence Before and After Reasoning in Language Modelscited by 1
  5. 2026From Sequence to Structure: Relational Uncertainty Propagation for LLM Agentscited by 1
  6. 2026Separating Capability from Confidence: Grounded Dual-State Calibration for GRPO-Trained Medical Vision-Language Modelscited by 0
  7. 2026Scored vs. Generated Readouts in Behavioral Language Models: An Empirical Study of Elicitation Formatcited by 0
  8. 2026On the Pitfalls of Verbalized Confidence Priors for Calibrating Large Reasoning Modelscited by 0
  9. 2026Calibration, Not Answer Selection: Distilling Internal Confidence in Reasoning Modelscited by 0
  10. 2026Shared Experience, Separate Learning: Companion Confidence Calibration for LLMscited by 0
  11. 2026When Words Fall Short: Iterative Synergy Between Verbalized Reasoning and Hidden Features for LLM Confidence Estimationcited by 0
  12. 2026Beyond Verbalized Confidence: Calibrating Reasoners with Differentiable Readoutscited by 0
2 more
  1. 2026OmniMed-Jev: Calibrating LVLM Confidence for Trustworthy Medical Multimodal Decisions via System Onecited by 0
  2. 2026Test-time Calibration Learning for Large Language Model Reasoningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Balancing Classification and Calibration Performance in Decision-Making LLMs via Calibration Aware Reinforcement Learning
  2. 2026On the effectiveness of reward functions in reinforcement learning for confidence calibration of large language models
  3. 2025ConfClip: Confidence-Weighted and Clipped Reward for Reinforcement Learning in LLMs
  4. 2026Learning from Failures: Correction-Oriented Policy Optimization with Verifiable Rewards
  5. 2025FAPO: Flawed-Aware Policy Optimization for Efficient and Reliable Reasoning
  6. 2026Overconfident Errors Need Stronger Correction: Asymmetric Confidence Penalties for Reinforcement Learning
  7. 2026ConSteer-RL: Steering Reasoning Capabilities in Large Language Models via Confidence-Aware Reinforcement Learning
  8. 2025Position: The Hidden Costs and Measurement Gaps of Reinforcement Learning with Verifiable Rewards