mlsanityData to 14 Oct 2026

CriterAlign: Criterion-Centric Rationale Alignment for Code Preference Judging

arXiv 2605.19665 ↗First posted 19 May 2026cs.SE cs.AIcited by 0

On the map: LLM alignment / RLHF / preference optimisation → Process reward models for reasoning

Abstract (arXiv)

Pairwise human preference prediction is central to evaluating code-generation systems, where quality often depends on task-specific trade-offs beyond functional correctness. While rubric-based LLM judges improve interpretability by decomposing evaluation into explicit criteria, most existing pipelines remain pointwise: they score each response independently and derive preferences by comparing aggregated scores. We show that this design is poorly matched to pairwise code preference prediction and can underperform a strong monolithic judge. We propose CriterAlign, a criterion-centric framework that adapts rubric-based judging to pairwise preference evaluation through direct criterion-level pairwise judgments, tie-driven criterion refinement, swap-consistency filtering, and final pairwise synthesis. We further introduce Human-Preference-Aligned Guidance (HPAG), synthesized offline from training examples by extracting recurring rationale gaps between human preferences and monolithic judge predictions, and injected into the criterion generator, criterion judge, and final judge. On BigCodeReward, CriterAlign improves a Qwen2.5-VL-32B monolithic judge from 60.4% to 66.3% accuracy, with ablations confirming the contributions of pairwise criterion design and HPAG.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Process reward models for reasoning, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 45 of its 67 references that are in the map. First posted on arXiv 19 May 2026.

Builds on

45 of 67 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025BigCodeArena: Unveiling More Reliable Human Preferences in Code Generation via Execution9 Oct
  2. 2024Generative Verifiers: Reward Modeling as Next-Token Prediction27 Aug

Other references · 43

  1. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  2. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 5,944
  3. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,373
  4. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  5. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  6. 2024A Survey on LLM-as-a-Judgecited by 1,423
  7. 2023Prometheus: Inducing Fine-grained Evaluation Capability in Language Modelscited by 575
  8. 2024Large Language Models: A Surveycited by 525
  9. 2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 465
  10. 2024Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Modelscited by 446
  11. 2025Kimi-VL Technical Reportcited by 388
  12. 2023JudgeLM: Fine-tuned Large Language Models are Scalable Judgescited by 367
31 more
  1. 2025Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domainscited by 366
  2. 2024Self-Preference Bias in LLM-as-a-Judgecited by 258
  3. 2025Inference-Time Scaling for Generalist Reward Modelingcited by 239
  4. 2025OpenRubrics: Towards Scalable Synthetic Rubric Generation for Reward Modeling and LLM Alignmentcited by 132
  5. 2024Critique-out-Loud Reward Modelscited by 106
  6. 2024Trust or Escalate: LLM Judges with Provable Guarantees for Human Agreementcited by 105
  7. 2024TICKing All the Boxes: Generated Checklists Improve LLM Evaluation and Generationcited by 68
  8. 2026Rethinking Rubric Generation for Improving LLM Judge and Reward Modeling for Open-ended Taskscited by 62
  9. 2024CodeUltraFeedback: An LLM-as-a-Judge Dataset for Aligning Large Language Models to Coding Preferencescited by 57
  10. 2023SALMON: Self-Alignment with Instructable Reward Modelscited by 55
  11. 2025Chasing the Tail: Effective Rubric-based Reward Modeling for Large Language Model Post-Trainingcited by 53
  12. 2025Online Rubrics Elicitation from Pairwise Comparisonscited by 46
  13. 2024CodeJudge: Evaluating Code Generation with Large Language Modelscited by 44
  14. 2024Inverse Constitutional AI: Compressing Preferences into Principlescited by 41
  15. 2024HD-Eval: Aligning Large Language Model Evaluators Through Hierarchical Criteria Decompositioncited by 33
  16. 2025CodeJudgeBench: Benchmarking LLM-as-a-Judge for Coding Taskscited by 33
  17. 2025MCTS-Judge: Test-Time Scaling in LLM-as-a-Judge for Code Correctness Evaluationcited by 32
  18. 2024CodeJudge-Eval: Can Large Language Models be Good Judges in Code Understanding?cited by 30
  19. 2025Rubric Is All You Need: Enhancing LLM-based Code Evaluation With Question-Specific Rubricscited by 29
  20. 2026Open Rubric System: Scaling Reinforcement Learning with Pairwise Adaptive Rubriccited by 27
  21. 2025Copilot Arena: A Platform for Code LLM Evaluation in the Wildcited by 26
  22. 2026Autorubric: A Unifying Framework for Rubric-Based LLM Evaluation on Non-Verifiable Taskscited by 26
  23. 2024Learning Code Preference via Synthetic Evolutioncited by 19
  24. 2026Learning to Judge: LLMs Designing and Applying Evaluation Rubricscited by 19
  25. 2025Don't Judge Code by Its Cover: Exploring Biases in LLM Judges for Code Evaluationcited by 18
  26. 2026CDRRM: Contrast-Driven Rubric Generation for Reliable and Interpretable Reward Modelingcited by 12
  27. 2025WebDevJudge: Evaluating (M)LLMs as Critiques for Web Development Qualitycited by 11
  28. 2025CodeVisionary: An Agent-based Framework for Evaluating Large Language Models in Code Generationcited by 6
  29. 2024User Centric Evaluation of Code Generation Toolscited by 5
  30. 2026iRULER: Intelligible Rubric-Based User-Defined LLM Evaluation for Revisioncited by 3
  31. 2026Comparing Developer and LLM Biases in Code Evaluationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Learning to Align Human Code Preferences
  2. 2025Teaching Your Models to Understand Code via Focal Preference Alignment
  3. 2026Bias in the Loop: Auditing LLM-as-a-Judge for Software Engineering
  4. 2024Evaluating and Aligning CodeLLMs on Human Preference
  5. 2026Subjective Code Preferences in Experts and Large Language Models ↗
  6. 2022Aligning Offline Metrics and Human Judgments of Value for Code Generation Models
  7. 2026Interaction-as-a-Judge
  8. 2025CODE-DITING: A Reasoning-Based Metric for Functional Alignment in Code Evaluation