# Are We on the Right Way to Assessing LLM-as-a-Judge?

arXiv: https://arxiv.org/abs/2512.16041 · HTML: https://mlsanity.com/paper/2512.16041/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Dec 2025.

## Built on by (most cited first, 24 of 24)

- [UniCorn: Towards Self-Improving Unified Multimodal Models through Self-Generated Supervision](https://mlsanity.com/paper/2601.03193/) (2026)
- [Grading Scale Impact on LLM-as-a-Judge: Human-LLM Alignment Is Highest on 0-5 Grading Scale](https://mlsanity.com/paper/2601.03444/) (2026)
- [JudgeSense: A Benchmark for Prompt Sensitivity in LLM-as-a-Judge Systems](https://mlsanity.com/paper/2604.23478/) (2026)
- [Time to REFLECT: Can We Trust LLM Judges for Evidence-based Research Agents?](https://mlsanity.com/paper/2605.19196/) (2026)
- [Toward Robust LLM-Based Judges: Taxonomic Bias Evaluation and Debiasing Optimization](https://mlsanity.com/paper/2603.08091/) (2026)
- [ChartEditBench: Evaluating Grounded Multi-Turn Chart Editing in Multimodal Language Models](https://mlsanity.com/paper/2602.15758/) (2026)
- [Aletheia: What Makes RLVR For Code Verifiers Tick?](https://mlsanity.com/paper/2601.12186/) (2026)
- [From Holistic Evaluation to Structured Criteria: Rubrics Across the Evolving LLM Landscape](https://mlsanity.com/paper/2606.08625/) (2026)
- [MM-tau-p2: Persona-Adaptive Prompting for Robust Multi-Modal Agent Evaluation in Dual-Control Settings](https://mlsanity.com/paper/2603.09643/) (2026)
- [To See is Not to Master: Teaching LLMs to Use Private Libraries for Code Generation](https://mlsanity.com/paper/2603.15159/) (2026)
- [Understanding LLM Evaluator Behavior: A Structured Multi-Evaluator Framework for Merchant Risk Assessment](https://mlsanity.com/paper/2602.05110/) (2026)
- [LogicGraph : Benchmarking Multi-Path Logical Reasoning via Neuro-Symbolic Generation and Verification](https://mlsanity.com/paper/2602.21044/) (2026)
- [Themis: Training Robust Multilingual Code Reward Models for Flexible Multi-Criteria Scoring](https://mlsanity.com/paper/2605.00754/) (2026)
- [Framing, Judging, Steering: An Assessable Competency Model for Teach-ing Students to Reason With Generative AI](https://mlsanity.com/paper/2606.05983/) (2026)
- [PaperJury: Due-Process Review for Bounded LaTeX Revision](https://mlsanity.com/paper/2606.16322/) (2026)
- [Pair Difficulty Matters: Rethinking Pairwise LLM-as-a-Judge Evaluation and Consistency](https://mlsanity.com/paper/2609.37577/) (2026)
- [Triviality Corrected Endogenous Reward](https://mlsanity.com/paper/2604.11522/) (2026)
- [Policy-Grounded Dynamic Facet Suggestions for Job Search](https://mlsanity.com/paper/2605.16479/) (2026)
- [LLM-Derived Preference Judgments Are Not Self-Consistent](https://mlsanity.com/paper/2608.17644/) (2026)
- [Beyond Correctness: Validity-Oriented Evaluation of Biomedical LLM Judges](https://mlsanity.com/paper/2608.29127/) (2026)
- [FailBench: How Reliable are VLMs at Judging Robot Task Success?](https://mlsanity.com/paper/2609.03611/) (2026)
- [LLJ Cards: Best practices for the Use of LLMs as Judges](https://mlsanity.com/paper/2609.24516/) (2026)
- [From Judgment Quality to Downstream Utility: Rethinking LLM-as-a-Judge for Open-Ended Tasks](https://mlsanity.com/paper/2609.37145/) (2026)
- [Understanding Machine Taste: Aesthetic Agreement Across Foundation Models in K-pop](https://mlsanity.com/paper/2610.17731/) (2026)

## Abstract (arXiv)

LLM-as-a-Judge has been widely adopted as an evaluation method and served as supervised rewards in model training. However, existing benchmarks for LLM-as-a-Judge are mainly relying on human-annotated ground truth, which introduces human bias that undermines the assessment of reliability and imposes scalability constraints. To overcome these limitations, we introduce Sage, a novel evaluation suite that assesses the quality of LLM judges without necessitating any human annotation. Inspired by axioms of rational choice theory, Sage introduces two new lenses for measuring LLM-as-a-Judge: local self-consistency (pair-wise preference stability) and global logical consistency (transitivity across a full set of preferences). We curate a dataset of 650 questions by combining structured benchmark problems with real-world user queries. Our experiments demonstrate both the stability of our metrics and their high correlation with supervised benchmarks like LLMBar and RewardBench2, confirming Sage's reliability as an evaluation suite for the robustness and accuracy of LLM-as-a-Judge. Based on Sage, we reveal that current state-of-the-art LLMs exhibit significant reliability problems when acting as judges in both scoring and pairwise settings; even the top-performing models, Gemini-2.5-Pro and GPT-5, fail to maintain consistent preferences in nearly a quarter of difficult cases. We attribute this to a new phenomenon called situational preference, which explains why explicit rubrics or criteria can help the model judge consistently across answer pairs. Our further analysis shows that finetuned LLM-as-a-Judge is a feasible method to boost performance, and the panel-based judge as well as deep reasoning can enhance the judging consistency. We also find substantial inconsistency in human judgments, which indicates that human annotation may not be a reliable gold standard.