mlsanityData to 7 Oct 2026

MatrixReward: Reward from Rubric Matrix for Open-Ended Generation

arXiv 2610.00389 ↗First posted 30 Sep 2026cs.LG cs.AI stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

Open-ended query generation lacks standard answers, thus necessitating an effective reward mechanism. Pointwise scoring rubrics provide limited information about the relative quality of sample answers under the same prompt; merging multiple rubric judgments into a single score may also mask the differences between these answers. We propose MatrixReward, which constructs rewards from a rollout-by-rubric win-rate matrix obtained by comparing every pair of sampled responses under each rubric. The spread of each matrix column captures how strongly that rubric distinguishes the current rollouts, while correlations between columns reveal rubric repetition; together, these statistics yield data-dependent rubric weights. We combine these weights with the prior weights of rubrics. After column normalization and weighting, the observed per-rubric maxima and minima define positive and negative ideal profiles. Each rollout's distances to these two ideals determine its relative-closeness quality reward. Evaluated using Qwen3-8B on four open-ended query-answering benchmarks, MatrixReward achieves an average score of 63.02, outperforming the strongest baseline by approximately 2.0%. These results support the idea that matrices derived from relative comparisons can be used to construct rewards more reasonably for open-ended generative reinforcement learning.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 27 of its 32 references that are in the map. First posted on arXiv 30 Sep 2026.

Builds on

27 of 32 references.

References · 27

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  3. 2025Qwen3 Technical Reportcited by 8,195
  4. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,678
  5. 2025Understanding R1-Zero-Like Training: A Critical Perspectivecited by 1,276
  6. 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 877
  7. 2024Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluatorscited by 865
  8. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 738
  9. 2024From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipelinecited by 484
  10. 2023UltraFeedback: Boosting Language Models with Scaled AI Feedbackcited by 394
  11. 2025Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domainscited by 328
  12. 2024WildBench: Benchmarking LLMs with Challenging Tasks from Real Users in the Wildcited by 211
15 more
  1. 2026GDPO: Group reward-Decoupled Normalization Policy Optimization for Multi-reward RL Optimizationcited by 175
  2. 2025OpenRubrics: Towards Scalable Synthetic Rubric Generation for Reward Modeling and LLM Alignmentcited by 118
  3. 2025WritingBench: A Comprehensive Benchmark for Generative Writingcited by 95
  4. 2025Chasing the Tail: Effective Rubric-based Reward Modeling for Large Language Model Post-Trainingcited by 46
  5. 2026Open Rubric System: Scaling Reinforcement Learning with Pairwise Adaptive Rubriccited by 26
  6. 2025Pairwise or Pointwise? Evaluating Feedback Protocols for Bias in LLM-Based Evaluationcited by 23
  7. 2026ArenaRL: Scaling RL for Open-Ended Agents via Tournament-based Relative Rankingcited by 17
  8. 2026Not Every Rubric Teaches Equally: Policy-Aware Rubric Rewards for RLVRcited by 9
  9. 2026Tournament-GRPO: Group-Wise Tournament Rewards for Reinforcement Learning in Open-Ended Long-Form Generationcited by 7
  10. 2026GOPO: Policy Optimization using Ranked Rewardscited by 5
  11. 2026SAW: Stage-Aware Dynamic Weighting for Multi-Objective Reinforcement Learning in Large Language Modelscited by 3
  12. 2026GD2PO: Mitigating Multi-Reward Conflicts via Group-Dynamic reward-Decoupled Policy Optimizationcited by 3
  13. 2026RRC: Unlocking Generative Reward Models in LLM Reinforcement Learning via Ranking-Based Reward Constructioncited by 2
  14. 2026Learn What's Left, Not What's Mastered: Saturation Aware Advantage Reweighting for Multi-Reward Policy Optimizationcited by 2
  15. 2026Uncovering and Mitigating Aggregation-Induced Reward Hacking in Multi-Reward Reinforcement Learningcited by 2

Built on by

Not cited yet.