mlsanityData to 14 Oct 2026

Co-rewarding: Stable Self-supervised RL for Eliciting Reasoning in Large Language Models

arXiv 2508.00410 ↗First posted 1 Aug 2025cs.LGcited by 23

On the map: LLM reasoning → Reinforcement learning for reasoning

Abstract (arXiv)

While reinforcement learning with verifiable rewards (RLVR) is effective to improve the reasoning ability of large language models (LLMs), its reliance on human-annotated labels leads to the scaling up dilemma, especially for complex tasks. Recent self-rewarding methods investigate a label-free alternative to unlock the reasoning capabilities of LLMs, yet they frequently encounter the non-negligible training collapse issue, as the single-view supervision signal easily forms the self-consistent illusion, yielding the reward hacking. Inspired by the success of self-supervised learning, we propose textitCo-rewarding, a novel self-supervised RL framework that improves training stability by seeking complementary supervision from another views. Specifically, we instantiate Co-rewarding in two ways: (1) textitCo-rewarding-I is a data-side instantiation that derives reward signals from contrastive agreement across semantically analogous questions; and (2) textitCo-rewarding-II is a model-side instantiation that maintains a slowly-updated reference teacher with pseudo labels to realize self-distillation. Intuitively, such instantiations introduce different levels of discrepancy to increase the difficulty of training collapse on trivial reasoning solutions. We also explore their orthogonally combined version to further boost the performance. Empirically, Co-rewarding exhibits stable training across various setups, and outperforms other self-rewarding baselines by +3.31% improvements on average on multiple mathematical reasoning benchmarks, especially by +7.49% on Llama-3.2-3B-Instruct. Notably, Co-rewarding reaches or even surpasses RLVR with ground-truth (GT) label in several cases, such as a Pass@1 of 94.01% on GSM8K with Qwen3-8B-Base remarkably higher than GT. Our code is released at https://github.com/tmlr-group/Co-rewarding.

Key facts. Cited by 23 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 46 of its 83 references that are in the map. First posted on arXiv 1 Aug 2025.

Builds on

46 of 83 references.

Closest ancestors · 23

Same direction, newest first

  1. 2025No Free Lunch: Rethinking Internal Feedback for LLM Reasoning20 Jun
  2. 2025Consistent Paths Lead to Truth: Self-Rewarding Reinforcement Learning for LLM Reasoning10 Jun
  3. 2025Confidence Is All You Need: Few-Shot RL Fine-Tuning of Language Models5 Jun
  4. 2025Maximizing Confidence Alone Improves Reasoning28 May
  5. 2025Can Large Reasoning Models Self-Train?27 May
  6. 2025Learning to Reason without External Rewards26 May
  7. 2025SeRL: Self-Play Reinforcement Learning for Large Language Models with Limited Data25 May
  8. 2025The Unreasonable Effectiveness of Entropy Minimization in LLM Reasoning21 May
  9. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Data6 May
  10. 2025Reinforcement Learning for Reasoning in Large Language Models with One Training Example29 Apr
  11. 2025TTRL: Test-Time Reinforcement Learning22 Apr
  12. 2025Right Question is Already Half the Answer: Fully Unsupervised LLM Reasoning Incentivization8 Apr
11 more
  1. 2025GPG: A Simple and Strong Reinforcement Learning Baseline for Model Reasoning3 Apr
  2. 2025CPPO: Accelerating the Training of Group Relative Policy Optimization-Based Reasoning Models28 Mar
  3. 2025Understanding R1-Zero-Like Training: A Critical Perspective26 Mar
  4. 2025Reinforcement Learning for Reasoning in Small LLMs: What Works and What Doesn't20 Mar
  5. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scale18 Mar
  6. 2025R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimization17 Mar
  7. 2025Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learning20 Feb
  8. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning22 Jan
  9. 2025REINFORCE++: Stabilizing Critic-Free Policy Optimization with Global Advantage Normalization4 Jan
  10. 2024KTO: Model Alignment as Prospect Theoretic Optimization2 Feb
  11. 2022Solving math word problems with process- and outcome-based feedback25 Nov

Other references · 23

  1. 2023GPT-4 Technical Reportcited by 18,396
  2. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2025Qwen3 Technical Reportcited by 8,820
  5. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  6. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  7. 2024Qwen2.5 Technical Reportcited by 4,942
  8. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
  9. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  10. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  11. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,228
  12. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 665
11 more
  1. 2024ORPO: Monolithic Preference Optimization without Reference Modelcited by 562
  2. 2025Visual-RFT: Visual Reinforcement Fine-Tuningcited by 513
  3. 2024Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translationcited by 394
  4. 2024Direct Language Model Alignment from Online AI Feedbackcited by 260
  5. 2025R1-Zero's "Aha Moment" in Visual Reasoning on a 2B Non-SFT Modelcited by 165
  6. 2024Exploratory Preference Optimization: Harnessing Implicit Q*-Approximation for Sample-Efficient RLHFcited by 106
  7. 2024Binary Classifier Optimization for Large Language Model Alignmentcited by 45
  8. 2025Landscape of Thoughts: Visualizing the Reasoning Process of Large Language Modelscited by 25
  9. 2025Step-KTO: Optimizing Mathematical Reasoning through Stepwise Binary Feedbackcited by 22
  10. 2018Exploration versus exploitation in reinforcement learning: a stochastic control approachcited by 18
  11. 2025Two Minds Better Than One: Collaborative Reward Modeling for LLM Alignmentcited by 6

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 4; 2026: 19.

23 papers.

Sort:

By area:

  1. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 180
  2. 2025TraPO: A Semi-Supervised Reinforcement Learning Framework for Boosting LLM Reasoningcited by 7
  3. 2026Explainable LLM Unlearning Through Reasoningcited by 7
  4. 2026Reference-guided Policy Optimization for Molecular Optimization via LLM Reasoningcited by 6
  5. 2026When Self-Belief Misleads: Active Label Acquisition for Reinforcement Learning with Verifiable Rewardscited by 6
  6. 2026Scaling Large Reasoning Models beyond Human Supervision: A Path toward Superintelligencecited by 5
  7. 2025Towards Understanding Valuable Preference Data for Large Language Model Alignmentcited by 4
  8. 2025Breaking the Self-Confirming Loop: Diagnosing and Mitigating Systemic Reward Bias in Self-Rewarding RLcited by 3
  9. 2026Distribution-Aware Reward Estimation for Test-Time Reinforcement Learningcited by 3
  10. 2026RewardFlow: Topology-Aware Reward Propagation on State Graphs for Agentic RL with Large Language Modelscited by 3
  11. 2026The Easy, the Hard, and the Learnable: Confidence and Difficulty-Adaptive Policy Optimization for LLM Reasoningcited by 2
  12. 2026Rate or Fate? RLV^varepsilonR: Reinforcement Learning with Verifiable Noisy Rewardscited by 1
11 more
  1. 2026Exploiting Verification-Generation Gap: Test-Time Reinforcement Learning with Confidence-Conditioned Verificationcited by 1
  2. 2026Learning What to Practice: Diagnosis-Guided Self-Evolution for Language Modelscited by 1
  3. 2026Two is better than one: A Collapse-free Multi-Reward RLIF Training Frameworkcited by 0
  4. 2026The Art of Interrogation: Consistency Amplifies Factuality in Spatial Reasoningcited by 0
  5. 2026CoRE: Consensus Rewards via Equilibrium for Test-Time Reinforcement Learningcited by 0
  6. 2026DE-Venus: A Data-Efficient RLVR Framework for Large Language Modelscited by 0
  7. 2026Do We Really Need KL Divergence for On-Policy Distillation of Large Language Models?cited by 0
  8. 2026When Updating Stops Being Learning: Rethinking LLM Self-Evolution via learnable information gaincited by 0
  9. 2026Group-Marginalized Self-Rewarding RL Drives Zero-Label Self-Evolvingcited by 0
  10. 2026Test-time Calibration Learning for Large Language Model Reasoningcited by 0
  11. 2026Checking the Checkers: A Synthesis of Verifier Noise and Robustness in RLVRcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Reinforcement Learning-based Semi-supervised Knowledge Distillation with LLM-as-a-Judge
  2. 2026SSL-R1: Self-Supervised Visual Reinforcement Post-Training for Multimodal Large Language Models
  3. 2025Enhancing the Outcome Reward-based RL Training of MLLMs with Self-Consistency Sampling
  4. 2026Co-RL: Unsupervised Reasoning Emerges from Diverse Cohort in Multi-agent RL
  5. 2026From RLVR to RLSVR: Task Transformation Induces Self-Verifiable Rewards for Open-Ended LLM Self-Improvement
  6. 2025RM-R1: Reward Modeling as Reasoning
  7. 2025SSL4RL: Revisiting Self-supervised Learning as Intrinsic Reward for Visual-Language Reasoning
  8. 2025Rewarding the Journey, Not Just the Destination: A Composite Path and Answer Self-Scoring Reward Mechanism for Test-Time Reinforcement Learning