mlsanityData to 14 Oct 2026

Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judges

arXiv 2406.12624 ↗First posted 18 Jun 2024cs.CL cs.AIcited by 205

Not on the map. Why?

Cited by 31 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Offering a promising solution to the scalability challenges associated with human evaluation, the LLM-as-a-judge paradigm is rapidly gaining traction as an approach to evaluating large language models (LLMs). However, there are still many open questions about the strengths and weaknesses of this paradigm, and what potential biases it may hold. In this paper, we present a comprehensive study of the performance of various LLMs acting as judges, focusing on a clean scenario in which inter-human agreement is high. Investigating thirteen judge models of different model sizes and families, judging answers of nine different 'examtaker models' - both base and instruction-tuned - we find that only the best (and largest) models achieve reasonable alignment with humans. However, they are still quite far behind inter-human agreement and their assigned scores may still differ with up to 5 points from human-assigned scores. In terms of their ranking of the nine exam-taker models, instead, also smaller models and even the lexical metric contains may provide a reasonable signal. Through error analysis and other studies, we identify vulnerabilities in judge models, such as their sensitivity to prompt complexity and length, and a tendency toward leniency. The fact that even the best judges differ from humans in this comparatively simple setup suggest that caution may be wise when using judges in more complex setups. Lastly, our research rediscovers the importance of using alignment metrics beyond simple percent alignment, showing that judges with high percent agreement can still assign vastly different scores.

Key facts. Cited by 205 papers in the map (data through 14 Oct 2026). Builds on 37 of its 47 references that are in the map. First posted on arXiv 18 Jun 2024.

Builds on

37 of 47 references.

References · 37

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2023GPT-4 Technical Reportcited by 18,396
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  4. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  5. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  7. 2023Mistral 7Bcited by 3,164
  8. 2021TruthfulQA: Measuring How Models Mimic Human Falsehoodscited by 2,975
  9. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,955
  10. 2023A Survey on Evaluation of Large Language Modelscited by 1,242
  11. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,184
  12. 2023Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Promptingcited by 1,122
25 more
  1. 2023Large Language Models are not Fair Evaluatorscited by 1,085
  2. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 734
  3. 2023JudgeLM: Fine-tuned Large Language Models are Scalable Judgescited by 367
  4. 2023Evaluating Large Language Models at Evaluating Instruction Followingcited by 302
  5. 2023FinanceBench: A New Benchmark for Financial Question Answeringcited by 270
  6. 2023Large Language Models Sensitivity to The Order of Options in Multiple-Choice Questionscited by 263
  7. 2023Generative Judge for Evaluating Alignmentcited by 169
  8. 2024Aligning with Human Judgement: The Role of Pairwise Preference in Large Language Model Evaluatorscited by 149
  9. 2024Who Validates the Validators? Aligning LLM-Assisted Evaluation of LLM Outputs with Human Preferencescited by 144
  10. 2023Verbosity Bias in Preference Labeling by Large Language Modelscited by 91
  11. 2023Are Large Language Model-based Evaluators the Solution to Scaling Up Multilingual Evaluation?cited by 88
  12. 2023Cognitive Mirage: A Review of Hallucinations in Large Language Modelscited by 84
  13. 2023LLM Comparative Assessment: Zero-shot NLG Evaluation through Pairwise Comparisons using Large Language Modelscited by 71
  14. 2023Style Over Substance: Evaluation Biases for Large Language Modelscited by 70
  15. 2023Can Large Language Models Understand Real-World Complex Instructions?cited by 69
  16. 2024The Benefits of a Concise Chain of Thought on Problem-Solving in Large Language Modelscited by 63
  17. 2024Quantifying Variance in Evaluation Benchmarkscited by 54
  18. 2023Evaluation Metrics in the Era of GPT-4: Reliably Evaluating Large Language Models on Sequence to Sequence Taskscited by 53
  19. 2023Evaluating Instruction-Tuned Large Language Models on Code Comprehension and Generationcited by 50
  20. 2023Improving Automatic VQA Evaluation Using Large Language Modelscited by 37
  21. 2024Are LLM-based Evaluators Confusing NLG Quality Criteria?cited by 36
  22. 2023Instruction-following Evaluation through Verbalizer Manipulationcited by 30
  23. 2023LLMEval: A Preliminary Study on How to Evaluate Large Language Modelscited by 23
  24. 2023WorldSense: A Synthetic Benchmark for Grounded Reasoning in Large Language Modelscited by 20
  25. 2024From Form(s) to Meaning: Probing the Semantic Depths of Language Models Using Multisense Consistencycited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 30; 2025: 78; 2026: 97.

205 papers.

Sort:

By area:

  1. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 494
  2. 2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 465
  3. 2024Agent-as-a-Judge: Evaluate Agents with Agentscited by 238
  4. 2025Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systemscited by 185
  5. 2025AbstentionBench: Reasoning LLMs Fail on Unanswerable Questionscited by 145
  6. 2025Preference Leakage: A Contamination Problem in LLM-as-a-judgecited by 125
  7. 2024Trust or Escalate: LLM Judges with Provable Guarantees for Human Agreementcited by 105
  8. 2024Systematic Evaluation of LLM-as-a-Judge in LLM Alignment Tasks: Explainable Metrics and Diverse Prompt Templatescited by 72
  9. 2024SafeBench: A Safety Evaluation Framework for Multimodal Large Language Modelscited by 69
  10. 2026Rethinking Rubric Generation for Improving LLM Judge and Reward Modeling for Open-ended Taskscited by 62
  11. 2024Archon: An Architecture Search Framework for Inference-Time Techniquescited by 60
  12. 2025One Token to Fool LLM-as-a-Judgecited by 60
38 more
  1. 2025No Free Labels: Limitations of LLM-as-a-Judge Without Human Groundingcited by 54
  2. 2024Limits to scalable evaluation at the frontier: LLM as Judge won't beat twice the datacited by 51
  3. 2025Multi-Attribute Steering of Language Models via Targeted Interventioncited by 46
  4. 2025Reinforcement Learning with Verifiable yet Noisy Rewards under Imperfect Verifierscited by 45
  5. 2025Evaluating Scoring Bias in LLM-as-a-Judgecited by 43
  6. 2024What is the Role of Small Models in the LLM Era: A Surveycited by 41
  7. 2025Answer Matching Outperforms Multiple Choice for Language Model Evaluationcited by 41
  8. 2025Rating Roulette: Self-Inconsistency in LLM-As-A-Judge Frameworkscited by 38
  9. 2026DeepResearch Bench II: Diagnosing Deep Research Agents via Rubrics from Expert Reportscited by 34
  10. 2024WildFeedback: Aligning LLMs With In-situ User Interactions And Feedbackcited by 31
  11. 2024Are LLM-Judges Robust to Expressions of Uncertainty? Investigating the effect of Epistemic Markers on LLM-based Evaluationcited by 29
  12. 2024JuStRank: Benchmarking LLM Judges for System Rankingcited by 29
  13. 2025Towards Reasoning Ability of Small Language Modelscited by 29
  14. 2025LLMs Can Generate a Better Answer by Aggregating Their Own Responsescited by 27
  15. 2025Safer or Luckier? LLMs as Safety Evaluators Are Not Robust to Artifactscited by 26
  16. 2025MultiLoKo: a multilingual local knowledge benchmark for LLMs spanning 31 languagescited by 26
  17. 2026UniCorn: Towards Self-Improving Unified Multimodal Models through Self-Generated Supervisioncited by 26
  18. 2026Nine Judges, Two Effective Votes: Correlated Errors Undermine LLM Evaluation Panelscited by 26
  19. 2024Reference-Guided Verdict: LLMs-as-Judges in Automatic Evaluation of Free-Form QAcited by 25
  20. 2026Noisy but Valid: Robust Statistical Evaluation of LLMs with Imperfect Judgescited by 24
  21. 2025The Illusion of Progress: Re-evaluating Hallucination Detection in LLMscited by 22
  22. 2024Reasoning Through Execution: Unifying Process and Outcome Rewards for Code Generationcited by 21
  23. 2025An Empirical Analysis of Uncertainty in Large Language Model Evaluationscited by 20
  24. 2025TrustJudge: Inconsistencies of LLM-as-a-Judge and How to Alleviate Themcited by 20
  25. 2025Cost-Optimal Active AI Model Evaluationcited by 19
  26. 2026Locomo-Plus: Beyond-Factual Cognitive Memory Evaluation Framework for LLM Agentscited by 19
  27. 2025Automatic Failure Attribution and Critical Step Prediction Method for Multi-Agent Systems Based on Causal Inferencecited by 18
  28. 2024PM-LLM-Benchmark: Evaluating Large Language Models on Process Mining Taskscited by 16
  29. 2024Bayesian Calibration of Win Rate Estimation with LLM Evaluatorscited by 16
  30. 2025Leveraging LLMs as Meta-Judges: A Multi-Agent Framework for Evaluating LLM Judgmentscited by 16
  31. 2025Beyond Consensus: Mitigating the Agreeableness Bias in LLM Judge Evaluationscited by 16
  32. 2026Diagnosing the Reliability of LLM-as-a-Judge via Item Response Theorycited by 16
  33. 2025MedArabiQ: Benchmarking Large Language Models on Arabic Medical Taskscited by 15
  34. 2026From Confident Closing to Silent Failure: Characterizing False Success in LLM Agentscited by 15
  35. 2024MIRAGE-Bench: Automatic Multilingual Benchmark Arena for Retrieval-Augmented Generation Systemscited by 14
  36. 2025Deterministic Inference across Tensor Parallel Sizes That Eliminates Training-Inference Mismatchcited by 14
  37. 2024Cognitive Overload Attack:Prompt Injection for Long Contextcited by 13
  38. 2026An Imperfect Verifier is Good Enough: Learning with Noisy Rewardscited by 13

50 most cited of 205 All 205 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Judge's Verdict: A Comprehensive Analysis of LLM Judge Capability Through Human Agreement
  2. 2026When Consistency Does Not Mean Reliability: Evaluating Local LLM Judges Against Human Ratings
  3. 2026Grading Scale Impact on LLM-as-a-Judge: Human-LLM Alignment Is Highest on 0-5 Grading Scale
  4. 2023TencentLLMEval: A Hierarchical Evaluation of Real-World Capabilities for Human-Aligned LLMs
  5. 2025On Evaluating LLM Alignment by Evaluating LLMs as Judges
  6. 2024From Calculation to Adjudication: Examining LLM judges on Mathematical Reasoning Tasks
  7. 2024Dissecting Human and LLM Preferences
  8. 2025The Alternative Annotator Test for LLM-as-a-Judge: How to Statistically Justify Replacing Human Annotators with LLMs