mlsanityData to 14 Oct 2026

OpenJev-RLCD: A Working RLCD Implementation

arXiv 2609.38850 ↗First posted 30 Sep 2026cs.AI cs.CLcited by 0

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Decision models such as Jev answer questions with probabilities, which are only useful if they are calibrated. Open-source reproductions rely on supervised fine-tuning plus temperature scaling, while reinforcement learning from verifiable rewards (RLVR) makes reasoning models overconfident. We present a working implementation of reinforcement learning for calibrated decisions (RLCD) for reasoning models: the model samples a rationale, and we score the answer distribution it commits to afterwards with a strictly proper scoring rule. A variance identity shows that scoring the mixture of several samples rewards disagreeing rationales, and that RLVR is exactly this mixture objective without its diversity term. Optimized naively, the per-rationale objective either switches reasoning off or is drowned out by policy-gradient noise, which leads to a two-stage recipe: calibrate, then reinforce. With Qwen3-1.7B on two reasoning tasks (3 seeds, paired tests), RLCD matches or beats SFT, RFT/STaR and GRPO (each temperature-scaled) in accuracy and beats all of them in selective prediction; on GSM8K answer verification a single query decides gvTwoCovFive% of the items at ≤5% error, versus gvGrpoCovFive% for GRPO. When uncertainty comes from annotator disagreement, RLCD provably cannot beat cross-entropy. Code and results: https://github.com/ZimmyGao/openjev-rlcd.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 13 of its 19 references that are in the map. First posted on arXiv 30 Sep 2026.

Builds on

13 of 19 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Beyond Binary Rewards: Training LMs to Reason About Their Uncertainty22 Jul

Other references · 12

  1. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2025Qwen3 Technical Reportcited by 8,820
  4. 2017On Calibration of Modern Neural Networkscited by 4,960
  5. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  6. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  7. 2017Selective Classification for Deep Neural Networkscited by 852
  8. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 812
  9. 2019Monte Carlo Gradient Estimation in Machine Learningcited by 283
  10. 2020What Can We Learn from Collective Human Opinions on Natural Language Inference Data?cited by 153
  11. 2024Language Models are Hidden Reasoners: Unlocking Latent Reasoning Capabilities via Self-Rewardingcited by 55
  12. 2023Training Chain-of-Thought via Latent-Variable Inferencecited by 47

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026JevAdvBench: A Benchmark and Black-Box Attacks for Reinforcement Learning for Calibrated Decisions Models
  2. 2025The Reasoning Boundary Paradox: How Reinforcement Learning Constrains Language Models
  3. 2025Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs
  4. 2026Know What You Know: Metacognitive Entropy Calibration for Verifiable RL Reasoning
  5. 2025RLPR: Extrapolating RLVR to General Domains without Verifiers
  6. 2025ConfClip: Confidence-Weighted and Clipped Reward for Reinforcement Learning in LLMs
  7. 2025Limits of Generalization in RLVR: Two Case Studies in Mathematical Reasoning
  8. 2025Position: The Hidden Costs and Measurement Gaps of Reinforcement Learning with Verifiable Rewards