mlsanityData to 14 Oct 2026

Neither Valid nor Reliable? Investigating the Use of LLMs as Judges

arXiv 2508.18076 ↗First posted 25 Aug 2025cs.CLcited by 39

Not on the map. Why?

Abstract (arXiv)

Evaluating natural language generation (NLG) systems remains a core challenge of natural language processing (NLP), further complicated by the rise of large language models (LLMs) that aims to be general-purpose. Recently, large language models as judges (LLJs) have emerged as a promising alternative to traditional metrics, but their validity remains underexplored. This position paper argues that the current enthusiasm around LLJs may be premature, as their adoption has outpaced rigorous scrutiny of their reliability and validity as evaluators. Drawing on measurement theory from the social sciences, we identify and critically assess four core assumptions underlying the use of LLJs: their ability to act as proxies for human judgment, their capabilities as evaluators, their scalability, and their cost-effectiveness. We examine how each of these assumptions may be challenged by the inherent limitations of LLMs, LLJs, or current practices in NLG evaluation. To ground our analysis, we explore three applications of LLJs: text summarization, data annotation, and safety alignment. Finally, we highlight the need for more responsible evaluation practices in LLJs evaluation, to ensure that their growing role in the field supports, rather than undermines, progress in NLG.

Key facts. Cited by 39 papers in the map (data through 14 Oct 2026). Builds on 84 of its 122 references that are in the map. First posted on arXiv 25 Aug 2025.

Builds on

84 of 122 references.

References · 84

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  4. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  5. 2022Constitutional AI: Harmlessness from AI Feedbackcited by 3,050
  6. 2022Emergent Abilities of Large Language Modelscited by 2,512
  7. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,276
  8. 2024HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusalcited by 1,429
  9. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,207
  10. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,184
  11. 2023LIMA: Less Is More for Alignmentcited by 1,100
  12. 2023Large Language Models are not Fair Evaluatorscited by 1,085
38 more
  1. 2022Red Teaming Language Models with Language Modelscited by 1,026
  2. 2023ChatGPT Outperforms Crowd-Workers for Text-Annotation Taskscited by 870
  3. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 734
  4. 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 686
  5. 2020SummEval: Re-evaluating Summarization Evaluationcited by 674
  6. 2023Bias and Fairness in Large Language Models: A Surveycited by 639
  7. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 505
  8. 2024From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipelinecited by 500
  9. 2024WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMscited by 496
  10. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 494
  11. 2023Large Language Models Are Not Robust Multiple Choice Selectorscited by 486
  12. 2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 465
  13. 2023Is ChatGPT a Good NLG Evaluator? A Preliminary Studycited by 422
  14. 2024Justice or Prejudice? Quantifying Biases in LLM-as-a-Judgecited by 368
  15. 2024Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Modelscited by 326
  16. 2024Humans or LLMs as the Judge? A Study on Judgement Biasescited by 323
  17. 2023Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervisioncited by 318
  18. 2023Large Language Models are Zero-Shot Rankers for Recommender Systemscited by 309
  19. 2019Measurement and Fairnesscited by 308
  20. 2021All That's 'Human' Is Not Gold: Evaluating Human Evaluation of Generated Textcited by 287
  21. 2023The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learningcited by 257
  22. 2024LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Taskscited by 243
  23. 2024ShieldGemma: Generative AI Content Moderation Based on Gemmacited by 243
  24. 2024SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Modelscited by 228
  25. 2021Societal Biases in Language Generation: Progress and Challengescited by 202
  26. 2020USR: An Unsupervised and Reference Free Evaluation Metric for Dialog Generationcited by 188
  27. 2023Time Travel in LLMs: Tracing Data Contamination in Large Language Modelscited by 177
  28. 2023AnnoLLM: Making Large Language Models to Be Better Crowdsourced Annotatorscited by 172
  29. 2023From Words to Watts: Benchmarking the Energy Costs of Large Language Model Inferencecited by 170
  30. 2023Generative Judge for Evaluating Alignmentcited by 169
  31. 2023Is ChatGPT better than Human Annotators? Potential and Limitations of ChatGPT in Explaining Implicit Hate Speechcited by 166
  32. 2023Benchmarking Cognitive Biases in Large Language Models as Evaluatorscited by 150
  33. 2020Re-evaluating Evaluation in Text Summarizationcited by 145
  34. 2024Optimization-based Prompt Injection Attack to LLM-as-a-Judgecited by 143
  35. 2023Investigating Data Contamination in Modern Benchmarks for Large Language Modelscited by 138
  36. 2024When Benchmarks are Targets: Revealing the Sensitivity of Large Language Model Leaderboardscited by 137
  37. 2025Aegis2.0: A Diverse AI Safety Dataset and Risks Taxonomy for Alignment of LLM Guardrailscited by 132
  38. 2024Is LLM-as-a-Judge Robust? Investigating Universal Adversarial Attacks on Zero-shot LLM Assessmentcited by 126

50 most cited of 84 All 84 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 9; 2026: 30.

39 papers.

Sort:

By area:

  1. 2025Large Language Models Hallucination: A Comprehensive Surveycited by 49
  2. 2026Nine Judges, Two Effective Votes: Correlated Errors Undermine LLM Evaluation Panelscited by 26
  3. 2026ResearchGym: Evaluating Language Model Agents on Real-World AI Researchcited by 22
  4. 2026WebCoderBench: Benchmarking Web Application Generation with Comprehensive and Interpretable Evaluation Metricscited by 18
  5. 2026Diagnosing the Reliability of LLM-as-a-Judge via Item Response Theorycited by 16
  6. 2025When Judgment Becomes Noise: How Design Failures in LLM Judge Benchmarks Silently Undermine Validitycited by 11
  7. 2026Hidden Measurement Error in LLM Pipelines Distorts Annotation, Evaluation, and Benchmarkingcited by 10
  8. 2026Judge Reliability Harness: Stress Testing the Reliability of LLM Judgescited by 8
  9. 2026Agreement Metrics for LLM-as-Judge Evaluation: What to Report and Whycited by 6
  10. 2026Expert Evaluation of Clinical AI Tools on Real Point-of-Care Clinical Queriescited by 5
  11. 2025AdvJudge-Zero: Binary Decision Flips in LLM-as-a-Judge via Adversarial Control Tokenscited by 4
  12. 2026LLM Judges Can Be Too Generous When There Is No Reference Answercited by 4
27 more
  1. 2025Modeling Motivated Reasoning in Law: Evaluating Strategic Role Conditioning in LLM Summarizationcited by 3
  2. 2026ComplexConstraints and Beyond: Expert Rubrics for RLVRcited by 3
  3. 2025Reward Models are Metrics in a Trench Coatcited by 2
  4. 2025ORCA: Open-ended Response Correctness Assessment for Audio Question Answeringcited by 2
  5. 2026Rethinking Hallucinations: Correctness, Consistency, and Prompt Multiplicitycited by 2
  6. 2026Augmenting Human Evaluation with LLM Judges: How Many Human Reviews Do You Need?cited by 2
  7. 2025Quantifying LLM Biases Across Instruction Boundary in Mixed Question Formscited by 1
  8. 2025KEO: Knowledge Extraction on OMIn via Knowledge Graphs and RAG for Safety-Critical Aviation Maintenancecited by 1
  9. 2026Evaluative Fingerprints: Stable and Systematic Differences in LLM Evaluator Behaviorcited by 1
  10. 2026LiveCultureBench: a Multi-Agent, Multi-Cultural Benchmark for Large Language Models in Dynamic Social Simulationscited by 1
  11. 2026When Scanners Lie: Evaluator Instability in LLM Red-Teamingcited by 1
  12. 2026Judge Circuits Explain Format-Induced Inconsistency in LLM-as-a-Judgecited by 1
  13. 2026LLM Judges Inconsistently Disagree Across Safety Criteria and Harm Categoriescited by 1
  14. 2026Challenges and Recommendations for LLM-as-a-Judge in Multilingual Settings and for Low-Resource Languagescited by 1
  15. 2025PolicyPad: Collaborative Prototyping of LLM Policiescited by 0
  16. 2026Leveraging Multimodal LLMs for Built Environment and Housing Attribute Assessment from Street-View Imagerycited by 0
  17. 2026GRASP: Deterministic argument ranking in interaction graphscited by 0
  18. 2026Naturalistic measure of social norms alignmentcited by 0
  19. 2026Reference-Free Evaluation of Reasoning in Open-Ended Question Answeringcited by 0
  20. 2026Even More Deception: Objective Misalignment in Mixed-Motive LLM Multi-Agent Systemscited by 0
  21. 2026An Investigation of the NeurIPS and ICML 2025 Position Trackscited by 0
  22. 2026OVIBench: Benchmarking Online Video Question Answering under Interruptioncited by 0
  23. 2026aipsy-judge: A Specialized, Psychologist-Corrected Local Judge for the Psychological Safety of Conversational AIcited by 0
  24. 2026Two Centuries of Sexism in British Parliament: A Computational Analysis of Women's Representation in the Hansard Corpuscited by 0
  25. 2026LLJ Cards: Best practices for the Use of LLMs as Judgescited by 0
  26. 2026How to Run Statistics over LLM Judges and Trust the Results: Calibrated Inference for Small-Sample AI Evaluation with evalstatscited by 0
  27. 2026SoK: Grading Open-Text Legal Answers, from Law School Exams to Trustworthy Rubric Judgescited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2023Exploring the Reliability of Large Language Models as Customized Evaluators for Diverse NLP Tasks
  2. 2026LLM as a Meta-Judge: Synthetic Data for NLP Evaluation Metric Validation
  3. 2023Through the Lens of Core Competency: Survey on Evaluation of Large Language Models
  4. 2024DnA-Eval: Enhancing Large Language Model Evaluation through Decomposition and Aggregation
  5. 2025The Alternative Annotator Test for LLM-as-a-Judge: How to Statistically Justify Replacing Human Annotators with LLMs
  6. 2026When Consistency Does Not Mean Reliability: Evaluating Local LLM Judges Against Human Ratings
  7. 2024Enhancing LLM Evaluations: The Garbling Trick
  8. 2024Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judges