mlsanityData to 14 Oct 2026

PACR: Progressively Ascending Confidence Reward for LLM Reasoning

arXiv 2510.22255 ↗First posted 25 Oct 2025cs.AI cs.CLcited by 9

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Reinforcement Learning with Verifiable Rewards (RLVR) has significantly improved LLM reasoning, but its sparse, outcome-based reward provides no guidance for intermediate steps, slowing exploration. We propose Progressively Ascending Confidence Reward (PACR), a dense, model-intrinsic reward computed directly from the model's evolving belief in the correct answer. PACR encodes the inductive bias that, along a well-formed reasoning trajectory, the probability of the ground-truth answer should have a generally ascending trend. We provide empirical and theoretical analysis validating that such an inductive bias constrains the exploration search space to regions richer in logically sound reasoning. We demonstrate that PACR accelerates exploration, reaches reward saturation with fewer trajectories, and yields improvements on multiple benchmarks. Our results suggest that dense, model-intrinsic shaping signals can make RLVR training more effective and reliable.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 17 of its 44 references that are in the map. First posted on arXiv 25 Oct 2025.

Builds on

17 of 44 references.

Closest ancestors · 10

Same direction, newest first

  1. 2025Beyond the First Error: Process Reward Models for Reflective Mathematical Reasoning20 May
  2. 2025Stop Summation: Min-Form Credit Assignment Is All Process Reward Model Needs for Reasoning21 Apr
  3. 2025Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model31 Mar
  4. 2025Understanding R1-Zero-Like Training: A Critical Perspective26 Mar
  5. 2025SimpleRL-Zoo: Investigating and Taming Zero Reinforcement Learning for Open Base Models in the Wild24 Mar
  6. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  7. 2025Process Reinforcement through Implicit Rewards3 Feb
  8. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning22 Jan
  9. 2024Free Process Rewards without Process Labels2 Dec
  10. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 7

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2025Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilitiescited by 3,796
  4. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  5. 2025gpt-oss-120b & gpt-oss-20b Model Cardcited by 1,386
  6. 2024Qwen2.5-Math Technical Report: Toward Mathematical Expert Model via Self-Improvementcited by 1,090
  7. 2025ProgRM: Build Better GUI Agents with Progress Rewardscited by 21

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 9.

9 papers.

  1. 2026Decomposed On-Policy Distillation for Vision-Language Reasoning: Steering Gradients for Visual Groundingcited by 10
  2. 2026TACO: Tool-Augmented Credit Optimization for Agentic Tool Usecited by 5
  3. 2026Know What You Know: Metacognitive Entropy Calibration for Verifiable RL Reasoningcited by 3
  4. 2026PDCR: Perception-Decomposed Confidence Reward for Vision-Language Reasoningcited by 3
  5. 2026Why Self-Rewarding Works: Theoretical Guarantees for Iterative Alignment of Language Modelscited by 1
  6. 2026Know Thy Reasoner: Not All Language Models Explore Alikecited by 0
  7. 2026Momentum for Reasoning: Dense Intrinsic Signals in Policy Optimizationcited by 0
  8. 2026QVal: Cheaply Evaluating Dense Supervision Signals for Long-Horizon LLM Agentscited by 0
  9. 2026When Should a VLM Look? Paying Only for Visual Calls That Were Needed and Usedcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025RLPR: Extrapolating RLVR to General Domains without Verifiers
  2. 2025ConfClip: Confidence-Weighted and Clipped Reward for Reinforcement Learning in LLMs
  3. 2025Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs
  4. 2026Grad2Reward: From Sparse Judgment to Dense Rewards for Improving Open-Ended LLM Reasoning
  5. 2025Consistent Paths Lead to Truth: Self-Rewarding Reinforcement Learning for LLM Reasoning
  6. 2025Beyond Correctness: Confidence-Aware Reward Modeling for Enhancing Large Language Model Reasoning
  7. 2026ERR+: Sequential Entropy Resolution for Efficient and Decisive LLM Reasoning
  8. 2026Likelihood-Based Reward Designs for General LLM Reasoning