mlsanityData to 7 Oct 2026

ConSteer-RL: Steering Reasoning Capabilities in Large Language Models via Confidence-Aware Reinforcement Learning

arXiv 2606.08088 ↗First posted 6 Jun 2026cs.LG cs.CLcited by 0

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Reinforcement Learning from Verifiable Rewards (RLVR) has recently become a key paradigm for improving the reasoning abilities of Large Language Models (LLMs), yet it remains limited by sparse binary rewards and its ignorance of model-internal uncertainty. In this paper, we propose ConSteer-RL, a simple yet effective framework that integrates token-level confidence signals derived from model log-probabilities into RLVR training. Specifically, building upon the Group Relative Policy Optimization (GRPO) framework, we construct a confidence-aware reward by aggregating per-token probabilities into a scalar confidence score and incorporating it into an awareness-based reward shaping mechanism that penalizes overconfident errors while reinforcing correct and confident reasoning. Experimental results demonstrate that ConSteer-RL consistently outperforms strong GRPO baselines, achieving average improvements of 2.3%-4.0% across different model scales.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 29 of its 42 references that are in the map. First posted on arXiv 6 Jun 2026.

Builds on

29 of 42 references.

Closest ancestors · 8

Same direction, newest first

  1. 2026Save the Good Prefix: Precise Error Penalization via Process-Supervised RL to Enhance LLM Reasoning26 Jan
  2. 2026GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimization8 Jan
  3. 2025Soft Adaptive Policy Optimization25 Nov
  4. 2025DeepMath-103K: A Large-Scale, Challenging, Decontaminated, and Verifiable Mathematical Dataset for Advancing Reasoning15 Apr
  5. 2025Understanding R1-Zero-Like Training: A Critical Perspective26 Mar
  6. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild24 Mar
  7. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning22 Jan
  8. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 21

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2025Qwen3 Technical Reportcited by 8,195
  3. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  4. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,601
  5. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,057
  6. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 795
  7. 2025Group Sequence Policy Optimizationcited by 788
  8. 2025Deep Think with Confidencecited by 199
  9. 2025Rewarding Doubt: A Reinforcement Learning Approach to Calibrated Confidence Expression of Large Language Modelscited by 48
  10. 2024Can AI Assistants Know What They Don't Know?cited by 47
  11. 2025VCR-Bench: A Comprehensive Evaluation Framework for Video Chain-of-Thought Reasoningcited by 41
  12. 2023Look Before You Leap: An Exploratory Study of Uncertainty Measurement for Large Language Modelscited by 40
9 more
  1. 2026Vision-DeepResearch Benchmark: Rethinking Visual and Textual Search for Multimodal Large Language Modelscited by 30
  2. 2025CodeRL+: Improving Code Generation via Reinforcement with Execution Semantics Alignmentcited by 22
  3. 2024Enhancing Confidence Expression in Large Language Models Through Learning from Past Experiencecited by 21
  4. 2025Agentic Jigsaw Interaction Learning for Enhancing Visual Perception and Reasoning in Vision-Language Modelscited by 11
  5. 2025V2P-Bench: Evaluating Video-Language Understanding with Visual Prompts for Better Human-Model Interactioncited by 10
  6. 2026PaperSearchQA: Learning to Search and Reason over Scientific Papers with RLVRcited by 5
  7. 2026From Verifiable Dot to Reward Chain: Harnessing Verifiable Reference-based Rewards for Reinforcement Learning of Open-ended Generationcited by 4
  8. 2026Beyond Binary: Turning Partial Success into Dense Verifiable Rewards for Reinforcement Learning in Code Generationcited by 3
  9. 2026VideoSeeker: Incentivizing Instance-level Video Understanding via Native Agentic Tool Invocationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025ConfClip: Confidence-Weighted and Clipped Reward for Reinforcement Learning in LLMs
  2. 2025Beyond Correctness: Confidence-Aware Reward Modeling for Enhancing Large Language Model Reasoning
  3. 2026Resource-Efficient Reinforcement for Reasoning Large Language Models via Dynamic One-Shot Policy Refinement
  4. 2026Process Supervision of Confidence Margin for Calibrated LLM Reasoning
  5. 2026On the effectiveness of reward functions in reinforcement learning for confidence calibration of large language models
  6. 2026Overconfident Errors Need Stronger Correction: Asymmetric Confidence Penalties for Reinforcement Learning
  7. 2026Decoupling Reasoning and Confidence: Resurrecting Calibration in Reinforcement Learning from Verifiable Rewards
  8. 2025RLPR: Extrapolating RLVR to General Domains without Verifiers