# TrustJudge: Inconsistencies of LLM-as-a-Judge and How to Alleviate Them

arXiv: https://arxiv.org/abs/2509.21117 · HTML: https://mlsanity.com/paper/2509.21117/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 48 references that are in the map.
- First posted on arXiv 25 Sep 2025.

## Builds on (most cited first, 30 of 42)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://mlsanity.com/paper/2501.12948/) (2025)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [Gemma 2: Improving Open Language Models at a Practical Size](https://mlsanity.com/paper/2408.00118/) (2024)
- [A Survey on Evaluation of Large Language Models](https://mlsanity.com/paper/2307.03109/) (2023)
- [Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators](https://mlsanity.com/paper/2404.04475/) (2024)
- [OpenAssistant Conversations -- Democratizing Large Language Model Alignment](https://mlsanity.com/paper/2304.07327/) (2023)
- [Self-Rewarding Language Models](https://mlsanity.com/paper/2401.10020/) (2024)
- [An Empirical Study of Catastrophic Forgetting in Large Language Models During Continual Fine-tuning](https://mlsanity.com/paper/2308.08747/) (2023)
- [From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipeline](https://mlsanity.com/paper/2406.11939/) (2024)
- [UltraFeedback: Boosting Language Models with Scaled AI Feedback](https://mlsanity.com/paper/2310.01377/) (2023)
- [MLLM-as-a-Judge: Assessing Multimodal LLM-as-a-Judge with Vision-Language Benchmark](https://mlsanity.com/paper/2402.04788/) (2024)
- [GPTScore: Evaluate as You Desire](https://mlsanity.com/paper/2302.04166/) (2023)
- [Humans or LLMs as the Judge? A Study on Judgement Biases](https://mlsanity.com/paper/2402.10669/) (2024)
- [JudgeBench: A Benchmark for Evaluating LLM-based Judges](https://mlsanity.com/paper/2410.12784/) (2024)
- [PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimization](https://mlsanity.com/paper/2306.05087/) (2023)
- [Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judges](https://mlsanity.com/paper/2406.12624/) (2024)
- [Meta-Rewarding Language Models: Self-Improving Alignment with LLM-as-a-Meta-Judge](https://mlsanity.com/paper/2407.19594/) (2024)
- [FLASK: Fine-grained Language Model Evaluation based on Alignment Skill Sets](https://mlsanity.com/paper/2307.10928/) (2023)
- [Mitigating the Alignment Tax of RLHF](https://mlsanity.com/paper/2309.06256/) (2023)
- [Benchmarking Foundation Models with Language-Model-as-an-Examiner](https://mlsanity.com/paper/2306.04181/) (2023)
- [Generative Judge for Evaluating Alignment](https://mlsanity.com/paper/2310.05470/) (2023)
- [Preference Leakage: A Contamination Problem in LLM-as-a-judge](https://mlsanity.com/paper/2502.01534/) (2025)
- [AutoSurvey: Large Language Models Can Automatically Write Surveys](https://mlsanity.com/paper/2406.10252/) (2024)
- [Branch-Solve-Merge Improves Large Language Model Evaluation and Generation](https://mlsanity.com/paper/2310.15123/) (2023)
- [Limitations of the LLM-as-a-Judge Approach for Evaluating LLM Outputs in Expert Knowledge Tasks](https://mlsanity.com/paper/2410.20266/) (2024)
- [LLM-Eval: Unified Multi-Dimensional Automatic Evaluation for Open-Domain Conversations with Large Language Models](https://mlsanity.com/paper/2305.13711/) (2023)
- [The Devil is in the Errors: Leveraging Large Language Models for Fine-grained Machine Translation Evaluation](https://mlsanity.com/paper/2308.07286/) (2023)

## Built on by (most cited first, 20 of 20)

- [SCOPE: Selective Conformal Optimized Pairwise LLM Judging](https://mlsanity.com/paper/2602.13110/) (2026)
- [Who can we trust? LLM-as-a-jury for Comparative Assessment](https://mlsanity.com/paper/2602.16610/) (2026)
- [LOGIGEN: Logic-Driven Generation of Verifiable Agentic Tasks](https://mlsanity.com/paper/2603.00540/) (2026)
- [Mediocrity is the key for LLM as a Judge Anchor Selection](https://mlsanity.com/paper/2603.16848/) (2026)
- [AutoPyVerifier: Learning Compact Executable Verifiers for Large Language Model Outputs](https://mlsanity.com/paper/2604.22937/) (2026)
- [Safeguarding LLM Agents from Misalignment through Provenance Analysis](https://mlsanity.com/paper/2607.01236/) (2026)
- [FairJudge: An Adaptive, Debiased, and Consistent LLM-as-a-Judge](https://mlsanity.com/paper/2602.06625/) (2026)
- [An Agentic Approach to Generating XAI-Narratives](https://mlsanity.com/paper/2603.20003/) (2026)
- [VLM-Guided Group Preference Alignment for Diffusion-based Human Mesh Recovery](https://mlsanity.com/paper/2602.19180/) (2026)
- [Agentic AI for Human Resources: LLM-Driven Candidate Assessment](https://mlsanity.com/paper/2603.26710/) (2026)
- [Offline A/B Testing of Slate Recommendation Systems with LLMs: Reducing the Dependency on Pre-Collected User Interaction Data](https://mlsanity.com/paper/2511.04541/) (2025)
- [LogicGraph : Benchmarking Multi-Path Logical Reasoning via Neuro-Symbolic Generation and Verification](https://mlsanity.com/paper/2602.21044/) (2026)
- [Enjoy Your Talk: A Human-Centered Benchmark for Multi-Turn Dialogue with Decoupled User Simulation, Target Modeling, and Judging](https://mlsanity.com/paper/2607.10428/) (2026)
- [Data-Centric Benchmarking of Exploit Generation in LLMs: Understanding the Impact of Fine-Tuning](https://mlsanity.com/paper/2606.15123/) (2026)
- [Prompt Perturbation for Reliable LLM Evaluation over Comparison Graphs](https://mlsanity.com/paper/2606.17634/) (2026)
- [Can LLMs Rank? A Tale of Triads and Triage](https://mlsanity.com/paper/2606.30412/) (2026)
- [TrustRoboReward: Preference-Ordered Isotonic Score Editing for Multi-Paradigm Robot Reward Models](https://mlsanity.com/paper/2608.08491/) (2026)
- [When Trust Meets Truth: Trust-Truth Separability in LLM-as-Judge](https://mlsanity.com/paper/2608.21097/) (2026)
- [En-ViMedNER: An English-Vietnamese Parallel Biomedical Corpus with UMLS Semantic Type Annotations](https://mlsanity.com/paper/2608.29890/) (2026)
- [Multiple latent orderings better predict language model preferences](https://mlsanity.com/paper/2609.22170/) (2026)

## Abstract (arXiv)

The adoption of Large Language Models (LLMs) as automated evaluators (LLM-as-a-judge) has revealed critical inconsistencies in current evaluation frameworks. We identify two fundamental types of inconsistencies: (1) Score-Comparison Inconsistency, where lower-rated responses outperform higher-scored ones in pairwise comparisons, and (2) Pairwise Transitivity Inconsistency, manifested through circular preference chains (A>B>C>A) and equivalence contradictions (A=B=Cneq A). We argue that these issues come from information loss in discrete rating systems and ambiguous tie judgments during pairwise evaluation. We propose TrustJudge, a probabilistic framework that addresses these limitations through two key innovations: 1) distribution-sensitive scoring that computes continuous expectations from discrete rating probabilities, preserving information entropy for more precise scoring, and 2) likelihood-aware aggregation that resolves transitivity violations using bidirectional preference probabilities or perplexity. We also formalize the theoretical limitations of current LLM-as-a-judge frameworks and demonstrate how TrustJudge's components overcome them. When evaluated with Llama-3.1-70B-Instruct as judge using our dataset, TrustJudge reduces Score-Comparison inconsistency by 8.43% (from 23.32% to 14.89%) and Pairwise Transitivity inconsistency by 10.82% (from 15.22% to 4.40%), while maintaining higher evaluation accuracy. Our work provides the first systematic analysis of evaluation framework inconsistencies in LLM-as-a-judge paradigms, offering both theoretical insights and practical solutions for reliable automated assessment. The framework demonstrates consistent improvements across various model architectures and scales, enabling more trustworthy LLM evaluation without requiring additional training or human annotations. The codes can be found at https://github.com/TrustJudge/TrustJudge.