mlsanityData to 14 Oct 2026

When Do Intrinsic Rewards Work for Code Reasoning? A Comprehensive Study

arXiv 2606.20881 ↗First posted 18 Jun 2026cs.AIcited by 0

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

Reinforcement learning with verifiable rewards (RLVR) has driven substantial progress in large language model reasoning, but relies on ground-truth supervision that is costly or infeasible, especially in coding tasks. Recent work addresses this by deriving rewards from a model's own signals, such as majority voting or confidence-based scores, achieving notable success on mathematical reasoning benchmarks. However, code generation poses distinct challenges: programs are structurally complex, semantically equivalent solutions may differ syntactically, and verification typically requires execution. Whether these intrinsic reward methods transfer effectively to code remains unexplored. In this work, we present a systematic empirical study of intrinsic reward methods for code generation. We conduct extensive experiments on LiveCodeBench, systematically evaluating representative certainty-based Reinforcement Learning from Internal Feedback (RLIF) approaches under different training scenarios and hyperparameter settings. Our experiments reveal that certainty-based methods yield early gains but inevitably collapse: models progressively shorten outputs and lose reasoning capability, with collapse speed sensitive to sample size and temperature. When used to initialize RLVR training, RLIF pre-training offers no significant improvement over training from scratch. We also provide actionable recommendations for using intrinsic rewards for training code reasoning models. Our study shows both the promise and limitations of intrinsic reward methods for code, informing future work on code models and agents.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 16 of its 16 references that are in the map. First posted on arXiv 18 Jun 2026.

Builds on

16 of 16 references.

Closest ancestors · 6

Same direction, newest first

  1. 2025Stabilizing Knowledge, Promoting Reasoning: Dual-Token Constraints for RLVR21 Jul
  2. 2025Consistent Paths Lead to Truth: Self-Rewarding Reinforcement Learning for LLM Reasoning10 Jun
  3. 2025Confidence Is All You Need: Few-Shot RL Fine-Tuning of Language Models5 Jun
  4. 2025Can Large Reasoning Models Self-Train?27 May
  5. 2025The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoning21 May
  6. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Training22 Nov

Other references · 10

  1. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  2. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  3. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  4. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,737
  5. 2022Competition-Level Code Generation with AlphaCodecited by 1,462
  6. 2025GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Modelscited by 467
  7. 2025Trae Agent: An LLM-based Agent for Software Engineering with Test-time Scalingcited by 94
  8. 2025ETTRL: Balancing Exploration and Exploitation in LLM Test-Time Reinforcement Learning Via Entropy Mechanismcited by 36
  9. 2025Post-Training Large Language Models via Reinforcement Learning from Self-Feedbackcited by 15
  10. 2025A Self-Supervised Reinforcement Learning Approach for Fine-Tuning Large Language Models Using Cross-Attention Signalscited by 7

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Improving Small Language Models for Code Generation with Reinforcement Learning from Verification Feedback
  2. 2025Spurious Rewards: Rethinking Training Signals in RLVR
  3. 2025Learning to Reason without External Rewards
  4. 2025Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs
  5. 2025Crossing the Reward Bridge: Expanding RL with Verifiable Rewards Across Diverse Domains
  6. 2025ConfClip: Confidence-Weighted and Clipped Reward for Reinforcement Learning in LLMs
  7. 2025No Free Lunch: Rethinking Internal Feedback for LLM Reasoning
  8. 2026From Verifiable Dot to Reward Chain: Harnessing Verifiable Reference-based Rewards for Reinforcement Learning of Open-ended Generation