# LegalRewardBench: Benchmarking Reward Models for Legal Preference Evaluation

arXiv: https://arxiv.org/abs/2610.18299 · HTML: https://mlsanity.com/paper/2610.18299/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Human values and AI feedback alignment, one of 10 directions in LLM alignment / RLHF / preference optimisation.
- Builds on 23 of its 33 references that are in the map.
- First posted on arXiv 13 Oct 2026.

## Placed in

- LLM alignment / RLHF / preference optimisation / [Human values and AI feedback alignment](https://mlsanity.com/alignment/human-values-and-ai-feedback-alignment/)

## Builds on (most cited first, 23 of 23)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Rubrics as Rewards: Reinforcement Learning Beyond Verifiable Domains](https://mlsanity.com/paper/2507.17746/) (2025)
- [Skywork-Reward: Bag of Tricks for Reward Modeling in LLMs](https://mlsanity.com/paper/2410.18451/) (2024)
- [Judging the Judges: A Systematic Study of Position Bias in LLM-as-a-Judge](https://mlsanity.com/paper/2406.07791/) (2024)
- [RLHF Workflow: From Reward Modeling to Online RLHF](https://mlsanity.com/paper/2405.07863/) (2024)
- [Self-Preference Bias in LLM-as-a-Judge](https://mlsanity.com/paper/2410.21819/) (2024)
- [RewardBench 2: Advancing Reward Model Evaluation](https://mlsanity.com/paper/2506.01937/) (2025)
- [SaulLM-7B: A pioneering Large Language Model for Law](https://mlsanity.com/paper/2403.03883/) (2024)
- [LawBench: Benchmarking Legal Knowledge of Large Language Models](https://mlsanity.com/paper/2309.16289/) (2023)
- [AgentRewardBench: Evaluating Automatic Evaluations of Web Agent Trajectories](https://mlsanity.com/paper/2504.08942/) (2025)
- [An Empirical Study of LLM-as-a-Judge for LLM Evaluation: Fine-tuned Judge Model is not a General Substitute for GPT-4](https://mlsanity.com/paper/2403.02839/) (2024)
- [RAG-RewardBench: Benchmarking Reward Models in Retrieval Augmented Generation for Preference Alignment](https://mlsanity.com/paper/2412.13746/) (2024)
- [Think-J: Learning to Think for Generative LLM-as-a-Judge](https://mlsanity.com/paper/2505.14268/) (2025)
- [LegalDelta: Enhancing Legal Reasoning in LLMs via Reinforcement Learning with Chain-of-Thought Guided Information Gain](https://mlsanity.com/paper/2508.12281/) (2025)
- [Unilaw-R1: A Large Language Model for Legal Reasoning with Reinforcement Learning and Iterative Inference](https://mlsanity.com/paper/2510.10072/) (2025)
- [IF-RewardBench: Benchmarking Judge Models for Instruction-Following Evaluation](https://mlsanity.com/paper/2603.04738/) (2026)
- [Toward Robust LLM-Based Judges: Taxonomic Bias Evaluation and Debiasing Optimization](https://mlsanity.com/paper/2603.08091/) (2026)
- [Legal Fact Prediction: The Missing Piece in Legal Judgment Prediction](https://mlsanity.com/paper/2409.07055/) (2024)
- [LegalOne: A Family of Foundation Models for Reliable Legal Reasoning](https://mlsanity.com/paper/2602.00642/) (2026)
- [RM-Distiller: Exploiting Generative LLM for Reward Model Distillation](https://mlsanity.com/paper/2601.14032/) (2026)
- [Building Legal Reward Models for Grounding and Abstention](https://mlsanity.com/paper/2609.14739/) (2026)

## Abstract (arXiv)

Reward models are central to preference alignment for legal tasks, yet existing reward model benchmarks are still primarily centered on general-domain preferences. In this work, we introduce LegalRewardBench, the first human-verified benchmark for evaluating reward models in the legal domain. The benchmark organizes legal preference evaluation into five capability levels: knowledge, understanding, analysis, decision, and writing, and preference examples are constructed through rubric-guided automatic judging combined with human verification. We evaluate three families of judges, covering both general and fine-tuned models, and generative and discriminative architectures. Experimental results show that strong general-purpose generative models are currently the most robust legal reward models, while existing fine-tuned reward models transfer less reliably to legal preference evaluation. We also conduct detailed comparisons of input organizations and training strategies.