mlsanityData to 7 Oct 2026

VI-CuRL: Stabilizing Verifier-Independent RL Reasoning via Confidence-Guided Variance Reduction

arXiv 2602.12579 ↗First posted 13 Feb 2026cs.LG cs.AIcited by 9

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Reinforcement Learning with Verifiable Rewards (RLVR) has emerged as a dominant paradigm for enhancing Large Language Models (LLMs) reasoning, yet its reliance on external verifiers limits its scalability. Recent findings suggest that RLVR primarily functions by eliciting latent capabilities, motivating the development of verifier-free algorithms. However, in such settings, standard methods like Group Relative Policy Optimization face a critical challenge: destructive gradient variance that often leads to training collapse. To address this issue, we introduce Verifier-Independent Curriculum Reinforcement Learning (VI-CuRL), a framework that leverages the model's intrinsic confidence to construct a curriculum independent from external verifiers. By prioritizing high-confidence samples, VI-CuRL effectively manages the bias-variance trade-off, specifically targeting the reduction of action and problem variance. We provide a rigorous theoretical analysis, proving that our estimator guarantees asymptotic unbiasedness. Empirically, VI-CuRL promotes stability and consistently outperforms verifier-dependent/independent baselines across math and general reasoning benchmarks with/without verifiers.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 15 of its 43 references that are in the map. First posted on arXiv 13 Feb 2026.

Builds on

15 of 43 references.

Closest ancestors · 10

Same direction, newest first

  1. 2025Efficient Reinforcement Learning with Semantic and Token Entropy for LLM Reasoning4 Dec
  2. 2025VCRL: Variance-based Curriculum Reinforcement Learning for Large Language Models24 Sep
  3. 2025No Free Lunch: Rethinking Internal Feedback for LLM Reasoning20 Jun
  4. 2025Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs17 Jun
  5. 2025Maximizing Confidence Alone Improves Reasoning28 May
  6. 2025The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models28 May
  7. 2025Reinforcing General Reasoning without Verifiers27 May
  8. 2025NOVER: Incentive Training for Language Models via Verifier-Free Reinforcement Learning21 May
  9. 2025Right Question is Already Half the Answer: Fully Unsupervised LLM Reasoning Incentivization8 Apr
  10. 2025Efficient Reinforcement Finetuning via Adaptive Curriculum Learning7 Apr

Other references · 5

  1. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 1,980
  2. 2022Language Models (Mostly) Know What They Knowcited by 1,783
  3. 2024Language Models are Hidden Reasoners: Unlocking Latent Reasoning Capabilities via Self-Rewardingcited by 53
  4. 2025Optimizing Chain-of-Thought Reasoners via Gradient Variance Minimization in Rejection Sampling and RLcited by 23
  5. 2025Robust Reinforcement Learning from Human Feedback for Large Language Models Fine-Tuningcited by 13

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 9.

9 papers.

  1. 2026Entropy trajectory shape predicts LLM reasoning reliability: A diagnostic study of uncertainty dynamics in chain-of-thoughtcited by 6
  2. 2026DyJR: Preserving Diversity in Reinforcement Learning with Verifiable Rewards via Dynamic Jensen-Shannon Replaycited by 3
  3. 2026SQL-ASTRA: Alleviating Sparse Feedback in Agentic SQL via Column-Set Matching and Trajectory Aggregationcited by 3
  4. 2026From P(y|x) to P(y): Investigating Reinforcement Learning in Pre-train Spacecited by 3
  5. 2026SocraticPO: Policy Optimization via Interactive Guidancecited by 3
  6. 2026VistaHop: Benchmarking Long-Horizon Visual DeepSearchcited by 1
  7. 2026Placing Puzzle Pieces Where They Matter: A Question Augmentation Framework for Reinforcement Learningcited by 0
  8. 2026Constraint-First Reasoning: A Training-Free Protocol for Exploiting Answer-Space Constraints in Mathematical Problem Solvingcited by 0
  9. 2026S3: Spectral Null-Space Swap Makes Reasoning Models Efficientcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025ConfClip: Confidence-Weighted and Clipped Reward for Reinforcement Learning in LLMs
  2. 2025VerifyBench: A Systematic Benchmark for Evaluating Reasoning Verifiers Across Domains
  3. 2026Curriculum Reinforcement Learning Can Incentivize Reasoning Capacity in LLMs Beyond the Base Model
  4. 2025RLPR: Extrapolating RLVR to General Domains without Verifiers
  5. 2026ConSteer-RL: Steering Reasoning Capabilities in Large Language Models via Confidence-Aware Reinforcement Learning
  6. 2025Extending RLVR to Open-Ended Tasks via Verifiable Multiple-Choice Reformulation
  7. 2025From Accuracy to Robustness: A Study of Rule- and Model-based Verifiers in Mathematical Reasoning
  8. 2026Learning from Failures: Correction-Oriented Policy Optimization with Verifiable Rewards