mlsanityData to 7 Oct 2026

Support Vector Rubrics: Closing the Gap Between Self-Generated and Human Rubrics

arXiv 2606.08077 ↗First posted 6 Jun 2026cs.CLcited by 1

On the map: Synthetic data and self-improvement for LLMs → Self-rewarding alignment and self-training

Abstract (arXiv)

Rubric-based evaluation is a promising paradigm for judging large language model (LLM) outputs, yet self-generated rubrics lag human-annotated criteria on hard instances. We argue this discriminative gap reflects an objective mismatch: self-generated rubrics describe good responses, whereas effective criteria must discriminate between close candidates. To close this gap, we introduce SVR (Support Vector Rubrics), a framework that recasts rubric construction as max-margin boundary learning over preference data. SVR mines contrastive features from preference pairs into a rubric bank, learns a prompt-conditioned selector together with global rubric weights, and iteratively refines the bank through support-pair selection and adversarial probing of hard negatives. At inference, given only the prompt, SVR retrieves the top-rubrics from the bank and scores responses. On RubricBench, SVR narrows the gap to human reference rubrics from 24.1 to 0.3 points and outperforms strong self-rubric and judge baselines, and the learned bank transfers across judges without retraining. On RewardBench 1&2, and RM-Bench, it remains competitive with dedicated reward models, demonstrating broader reward modeling capability. Overall, boundary-defining rubrics offer a principled route to closing the discriminative gap in LLM evaluation.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Self-rewarding alignment and self-training, one of 13 directions in Synthetic data and self-improvement for LLMs. Builds on 32 of its 46 references that are in the map. First posted on arXiv 6 Jun 2026.

Builds on

32 of 46 references.

References · 32

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  2. 2017Deep reinforcement learning from human preferencescited by 3,846
  3. 2015FaceNet: A Unified Embedding for Face Recognition and Clusteringcited by 2,833
  4. 2020Learning to summarize from human feedbackcited by 2,562
  5. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  6. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
  7. 2022Scaling Laws for Reward Model Overoptimizationcited by 1,056
  8. 2026DeepSeek-V4: Towards Highly Efficient Million-Token Context Intelligencecited by 877
  9. 2016Training Region-based Object Detectors with Online Hard Example Miningcited by 577
  10. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 480
  11. 2024Generative Verifiers: Reward Modeling as Next-Token Predictioncited by 465
  12. 2025Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domainscited by 328
20 more
  1. 2024Skywork-Reward: Bag of Tricks for Reward Modeling in LLMscited by 322
  2. 2023Reward Model Ensembles Help Mitigate Overoptimizationcited by 225
  3. 2025Skywork-Reward-V2: Scaling Preference Data Curation via Human-AI Synergycited by 204
  4. 2024RM-Bench: Benchmarking Reward Models of Language Models with Subtlety and Stylecited by 168
  5. 2025RM-R1: Reward Modeling as Reasoningcited by 149
  6. 2025RewardBench 2: Advancing Reward Model Evaluationcited by 136
  7. 2024HelpSteer2-Preference: Complementing Ratings with Preferencescited by 129
  8. 2025OpenRubrics: Towards Scalable Synthetic Rubric Generation for Reward Modeling and LLM Alignmentcited by 118
  9. 2024CheckEval: A reliable LLM-as-a-Judge framework for evaluating text generation using checklistscited by 87
  10. 2025HelpSteer3-Preference: Open Human-Annotated Preference Data across Diverse Tasks and Languagescited by 71
  11. 2024TICKing All the Boxes: Generated Checklists Improve LLM Evaluation and Generationcited by 62
  12. 2026Rethinking Rubric Generation for Improving LLM Judge and Reward Modeling for Open-ended Taskscited by 55
  13. 2025AdvancedIF: Rubric-Based Benchmarking and Reinforcement Learning for Advancing LLM Instruction Followingcited by 46
  14. 2026Alternating Reinforcement Learning for Rubric-Based Reward Modeling in Non-Verifiable LLM Post-Trainingcited by 45
  15. 2024LMUnit: Fine-grained Evaluation with Natural Language Unit Testscited by 32
  16. 2025Auto-Rubric: Learning From Implicit Weights to Explicit Rubrics for Reward Modelingcited by 29
  17. 2025RewardAnything: Generalizable Principle-Following Reward Modelscited by 27
  18. 2026Reinforcing Chain-of-Thought Reasoning with Self-Evolving Rubricscited by 23
  19. 2026RubricBench: Aligning Model-Generated Rubrics with Human Standardscited by 21
  20. 2026RubricEval: A Rubric-Level Meta-Evaluation Benchmark for LLM Judges in Instruction Followingcited by 19

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loopscited by 25

Similar, not cited

Close in wording, no citation either way

  1. 2026Rubrics as an Attack Surface: Stealthy Preference Drift in LLM Judges
  2. 2026Self-Preference Bias in Rubric-Based Evaluation of Large Language Models
  3. 2026Many Voices, One Reward: Multi-Role Rubric Generation for LLM Judging and Reward Modeling
  4. 2026GenRubric: Self-Evolving Rubric Generation for Scalable LLM Evaluation
  5. 2025Rubric-Conditioned LLM Grading: Alignment, Uncertainty, and Robustness
  6. 2026Preference-Aware Rubric Learning for Personalized Evaluation
  7. 2026CDRRM: Contrast-Driven Rubric Generation for Reliable and Interpretable Reward Modeling
  8. 2026Auto-Rubric as Reward: From Implicit Preferences to Explicit Multimodal Generative Criteria