mlsanityData to 14 Oct 2026

CoCA: Criterion-Aware Credit Assignment for Rubric-Based Reinforcement Learning

arXiv 2610.17733 ↗First posted 13 Oct 2026cs.LG cs.AIcited by 0

On the map: Deep reinforcement learning methods → RL training of language and agent models

Abstract (arXiv)

Rubric-based reinforcement learning (RL) evaluates responses against explicit criteria to construct rewards, offering a viable way to train large language models on open-ended tasks. However, standard methods compress criterion-level feedback into scalar rewards and assign uniform token advantages, leaving structured rubric supervision underutilized. Recent work seeks finer-grained guidance by treating rubrics as privileged information to derive self-distillation targets, but the resulting learning signal is not grounded in criterion outcomes and can lead to reward degradation. This motivates us to reframe the problem as criterion-aware credit assignment, where privileged rubric information serves as a relevance cue for credit allocation rather than a source of imitation targets. Accordingly, we propose CoCA, which derives criterion-level credits from group-relative outcomes and allocates them across tokens based on criterion-induced distribution shifts, yielding both outcome-grounded and fine-grained learning signals. Empirically, CoCA achieves an average gain of +4.2% over standard rubric-based RL across medicine and science benchmarks, demonstrating more effective learning on open-ended tasks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL training of language and agent models, one of 15 directions in Deep reinforcement learning methods. Builds on 46 of its 49 references that are in the map. First posted on arXiv 13 Oct 2026.

Builds on

46 of 49 references.

Closest ancestors · 8

Same direction, newest first

  1. 2026Rubric-to-Code Credit Assignment for Reinforcement Learning28 Aug
  2. 2026RISE-RL: Rubric-Informed Selective Exploration for Open-Ended Reinforcement Learning10 Aug
  3. 2026Alternating Reinforcement Learning with Contextual Rubric Rewards: Beyond the Scalarization Strategy4 Mar
  4. 2026Reinforcement Learning via Self-Distillation28 Jan
  5. 2026From Verifiable Dot to Reward Chain: Harnessing Verifiable Reference-based Rewards for Reinforcement Learning of Open-ended Generation26 Jan
  6. 2025DR Tulu: Reinforcement Learning with Evolving Rubrics for Deep Research24 Nov
  7. 2025Reinforcement Learning with Rubric Anchors18 Aug
  8. 2025Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domains23 Jul

Other references · 38

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2025Qwen3 Technical Reportcited by 8,820
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  4. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  5. 2024HybridFlow: A Flexible and Efficient RLHF Frameworkcited by 2,057
  6. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  7. 2020What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Examscited by 1,356
  8. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,345
  9. 2025Beyond the 80/20 Rule: High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoningcited by 559
  10. 2025Process Reinforcement through Implicit Rewardscited by 504
  11. 2026Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Modelscited by 455
  12. 2025HealthBench: Evaluating Large Language Models Towards Improved Human Healthcited by 429
26 more
  1. 2025A Survey of Reinforcement Learning for Large Reasoning Modelscited by 180
  2. 2018RUDDER: Return Decomposition for Delayed Rewardscited by 173
  3. 2025OpenRubrics: Towards Scalable Synthetic Rubric Generation for Reward Modeling and LLM Alignmentcited by 132
  4. 2025Checklists Are Better Than Reward Models For Aligning Language Modelscited by 128
  5. 2024LLM-Rubric: A Multidimensional, Calibrated Approach to Automated Evaluation of Natural Language Textscited by 87
  6. 2019Hindsight Credit Assignmentcited by 79
  7. 2026DeepSeek-V4.1-Flash: Pushing the Limits of KV Cache Compressioncited by 76
  8. 2025Writing-Zero: Bridge the Gap Between Non-verifiable Tasks and Verifiable Rewardscited by 50
  9. 2025ResearchQA: Evaluating Scholarly Question Answering at Scale Across 75 Fields with Survey-Mined Questions and Rubricscited by 49
  10. 2025LLMEval-Med: A Real-world Clinical Benchmark for Medical LLMs with Physician Validationcited by 34
  11. 2025Reward and Guidance through Rubrics: Promoting Exploration to Improve Multi-Domain Reasoningcited by 21
  12. 2025SciArena: An Open Evaluation Platform for Non-Verifiable Scientific Literature-Grounded Taskscited by 20
  13. 2026Rubric-Guided Self-Distillation: Post-Training Without Rubric Verifierscited by 15
  14. 2026Step-wise Rubric Rewards for LLM Reasoningcited by 14
  15. 2023Would I have gotten that reward? Long-term credit assignment by counterfactual contribution analysiscited by 13
  16. 2026Rethinking Reward Supervision: Rubric-Conditioned Self-Distillationcited by 13
  17. 2026Not Every Rubric Teaches Equally: Policy-Aware Rubric Rewards for RLVRcited by 11
  18. 2026Rubrics to Tokens: Bridging Response-level Rubrics and Token-level Rewards in Instruction Following Taskscited by 10
  19. 2025Beyond Pointwise Scores: Decomposed Criteria-Based Evaluation of LLM Responsescited by 8
  20. 2026DRACO: Fine-Grained Credit Assignment with Dynamic Rubrics for Long-Horizon Agent Trainingcited by 5
  21. 2026LLM-as-a-Coach: Experiential Learning for Non-Verifiable Taskscited by 4
  22. 2026Rubrics as Privileged Information for Open-Ended Generationcited by 4
  23. 2026Experience is the Best Teacher: Motivating Effective Exploration in Reinforcement Learning for LLMscited by 3
  24. 2026PaperGym: Rubric-Centered Evolution for Research-Plan Generationcited by 3
  25. 2026CoRT: Counterfactual Replay for Token-Level Rubric-Guided Policy Optimizationcited by 2
  26. 2026CARE: Contrastive Anchor-based Rubric Evolution for Large Language Model Post-Trainingcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Mitigating False Credit Propagation: Probabilistic Graphical Reward Aggregation for Rubric-Based Reinforcement Learning
  2. 2026OPD Before RL: Warm-Starting Rubric-Based RL with On-Policy Distillation
  3. 2026Small Language Models as Judges for Rubric-Based Reinforcement Learning
  4. 2026MetaRubric: Learning to Reward for Rubric-Based Reinforcement Learning
  5. 2026CriPO: Enhancing Rubric-based RL via Self-Distillation
  6. 2026Scoring Higher, Answering Worse: Mitigating Reward Hacking in Rubric-Based RL via Protocol-Level Rubrics
  7. 2026Rubric-Grounded RL: Structured Judge Rewards for Generalizable Reasoning
  8. 2026A Survey on Rubric-Guided Reinforcement Learning for Language Models