mlsanityData to 14 Oct 2026

Reinforcing Chain-of-Thought Reasoning with Self-Evolving Rubrics

arXiv 2602.10885 ↗First posted 11 Feb 2026cs.AI cs.LGcited by 26

On the map: LLM reasoning → Prompting and chain-of-thought reasoning

also in 2 more
Abstract (arXiv)

Despite chain-of-thought (CoT) playing crucial roles in LLM reasoning, directly rewarding it is difficult: training a reward model demands heavy human labeling efforts, and static RMs struggle with evolving CoT distributions and reward hacking. These challenges motivate us to seek an autonomous CoT rewarding approach that requires no human annotation efforts and can evolve gradually. Inspired by recent self-evolving training methods, we propose RLCER (Reinforcement Learning with CoT Supervision via Self-Evolving Rubrics), which enhances the outcome-centric RLVR by rewarding CoTs with self-proposed and self-evolving rubrics. We show that self-proposed and self-evolving rubrics provide reliable CoT supervision signals even without outcome rewards, enabling RLCER to outperform outcome-centric RLVR. Moreover, when used as in-prompt hints, these self-proposed rubrics further improve inference-time performance.

Key facts. Cited by 26 papers in the map (data through 14 Oct 2026). Placed in Prompting and chain-of-thought reasoning, one of 17 directions in LLM reasoning. Builds on 33 of its 37 references that are in the map. First posted on arXiv 11 Feb 2026.

Builds on

33 of 37 references.

Closest ancestors · 3

Same direction, newest first

  1. 2025What Characterizes Effective Reasoning? Revisiting Length, Review, and Structure of CoT23 Sep
  2. 2025What Makes a Good Reasoning Chain? Uncovering Structural Patterns in Long Chain-of-Thought Reasoning28 May
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Models28 Jan

Other references · 30

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2023Let's Verify Step by Stepcited by 3,789
  3. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  4. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  5. 2023A Survey of Large Language Modelscited by 2,576
  6. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 2,004
  7. 2022STaR: Bootstrapping Reasoning With Reasoningcited by 940
  8. 2017Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithmcited by 734
  9. 2024Self-Rewarding Language Modelscited by 681
  10. 2024Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Modelscited by 557
  11. 2025Process Reinforcement through Implicit Rewardscited by 504
  12. 2025Multi-Agent Collaboration Mechanisms: A Survey of LLMscited by 431
18 more
  1. 2025HealthBench: Evaluating Large Language Models Towards Improved Human Healthcited by 429
  2. 2025Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domainscited by 366
  3. 2025Absolute Zero: Reinforced Self-play Reasoning with Zero Datacited by 344
  4. 2025Demystifying Long Chain-of-Thought Reasoning in LLMscited by 326
  5. 2024ProcessBench: Identifying Process Errors in Mathematical Reasoningcited by 292
  6. 2025ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systemscited by 190
  7. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 180
  8. 2025DR Tulu: Reinforcement Learning with Evolving Rubrics for Deep Researchcited by 114
  9. 2025Reinforcement Learning with Rubric Anchorscited by 113
  10. 2025SPICE: Self-Play In Corpus Environments Improves Reasoningcited by 83
  11. 2025On the Interplay of Pre-Training, Mid-Training, and RL on Reasoning Language Modelscited by 69
  12. 2025Breaking the Exploration Bottleneck: Rubric-Scaffolded Reinforcement Learning for General LLM Reasoningcited by 63
  13. 2025Chasing the Tail: Effective Rubric-based Reward Modeling for Large Language Model Post-Trainingcited by 53
  14. 2025SeRL: Self-Play Reinforcement Learning for Large Language Models with Limited Datacited by 51
  15. 2025Chasing Moving Targets with Online Self-Play Reinforcement Learning for Safer Language Modelscited by 39
  16. 2025Search Self-play: Pushing the Frontier of Agent Capability without Supervisioncited by 34
  17. 2025Self-Rewarding Rubric-Based Reinforcement Learning for Open-Ended Reasoningcited by 11
  18. 2025The Alignment Waltz: Jointly Training Agents to Collaborate for Safetycited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 26.

26 papers.

Sort:

By area:

  1. 2026Agent-World: Scaling Real-World Environment Synthesis for Evolving General Agent Intelligencecited by 54
  2. 2026EvoRubrics: Dynamic Rubrics as Rewards via Adversarial Co-Evolution for LLM Reinforcement Learningcited by 14
  3. 2026EvoLM: Self-Evolving Language Models through Co-Evolved Discriminative Rubricscited by 13
  4. 2026PAPO: Stabilizing Rubric Integration Training via Decoupled Advantage Normalizationcited by 11
  5. 2026EvoRubric: Self-Evolving Rubric-Driven RL for Open-Ended Generationcited by 11
  6. 2026SCOPE: Self-Play via Co-Evolving Policies for Open-Ended Taskscited by 7
  7. 2026CoMAP: Co-Evolving World Models and Agent Policies for LLM Agentscited by 6
  8. 2026From Holistic Evaluation to Structured Criteria: Rubrics Across the Evolving LLM Landscapecited by 4
  9. 2026AgentStream: How Well Do Self-Evolving LLM Agents Perform Under Streaming Tasks?cited by 4
  10. 2026DeltaRubric: Generative Multimodal Reward Modeling via Joint Planning and Verificationcited by 3
  11. 2026AMARIS: A Memory-Augmented Rubric Improvement System for Rubric-Based Reinforcement Learningcited by 3
  12. 2026Claim-Level Rubric Rewards for Video Caption Reinforcement Learningcited by 3
14 more
  1. 2026Reinforcement Learning with Evolving Rubrics as Rewards for Audio Reasoningcited by 3
  2. 2026A Survey on Rubric-Guided Reinforcement Learning for Language Modelscited by 3
  3. 2026Rubric-Guided Process Reward for Stepwise Model Routingcited by 2
  4. 2026SERPO: Self-Evolving Rubric Policy Optimization for Open-Ended Test-Time Reinforcement Learningcited by 2
  5. 2026RubricReviewer: From Direct Critique to Objective and Comprehensive Rubric-Driven Peer Reviewcited by 2
  6. 2026Momentum-Coupled Rubric Adaptation for Detailed Image Captioningcited by 2
  7. 2026Support Vector Rubrics: Closing the Gap Between Self-Generated and Human Rubricscited by 1
  8. 2026RewardVerse: Rubric-Guided Policy Optimization for Video Reward Modelingcited by 1
  9. 2026Co-ReAct: Rubrics as Step-Level Collaborators for ReAct Agentscited by 0
  10. 2026Unlocking the Critic: Reward-Free Policy Optimization for LLM Post-Trainingcited by 0
  11. 2026From Judgment Quality to Downstream Utility: Rethinking LLM-as-a-Judge for Open-Ended Taskscited by 0
  12. 2026RISED: RubrIcs for agentic multi-environment Selection and sElf-Distillationcited by 0
  13. 2026MetaRubric: Learning to Reward for Rubric-Based Reinforcement Learningcited by 0
  14. 2026From Structure to Validity: Mechanism-Guided Generation and Refinement of LLM Rubricscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Rethinking Chain-of-Thought from the Perspective of Self-Training
  2. 2026S3-CoT: Self-Sampled Succinct Reasoning Enables Efficient Chain-of-Thought LLMs
  3. 2025Co-rewarding: Stable Self-supervised RL for Eliciting Reasoning in Large Language Models
  4. 2025Towards System 2 Reasoning in LLMs: Learning How to Think With Meta Chain-of-Thought
  5. 2024On the Hardness of Faithful Chain-of-Thought Reasoning in Large Language Models
  6. 2025From Reasoning to Super-Intelligence: A Search-Theoretic Perspective
  7. 2025The Challenge of Teaching Reasoning to LLMs Without RL or Distillation
  8. 2025Not All Thoughts are Generated Equal: Efficient LLM Reasoning via Multi-Turn Reinforcement Learning