mlsanityData to 7 Oct 2026

LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks

arXiv 2406.18403 ↗First posted 26 Jun 2024cs.CLcited by 236

Not on the map. Why?

Cited by 21 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

There is an increasing trend towards evaluating NLP models with LLMs instead of human judgments, raising questions about the validity of these evaluations, as well as their reproducibility in the case of proprietary models. We provide JUDGE-BENCH, an extensible collection of 20 NLP datasets with human annotations covering a broad range of evaluated properties and types of data, and comprehensively evaluate 11 current LLMs, covering both open-weight and proprietary models, for their ability to replicate the annotations. Our evaluations show substantial variance across models and datasets. Models are reliable evaluators on some tasks, but overall display substantial variability depending on the property being evaluated, the expertise level of the human judges, and whether the language is human or model-generated. We conclude that LLMs should be carefully validated against human judgments before being used as evaluators.

Key facts. Cited by 236 papers in the map (data through 7 Oct 2026). Builds on 48 of its 75 references that are in the map. First posted on arXiv 26 Jun 2024.

Builds on

48 of 75 references.

References · 48

  1. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  2. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  3. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  4. 2023G-Eval: NLG Evaluation using GPT-4 with Better Human Alignmentcited by 2,205
  5. 2018Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarizationcited by 1,308
  6. 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,151
  7. 2023Large Language Models are not Fair Evaluatorscited by 1,050
  8. 2015Teaching Machines to Read and Comprehendcited by 1,039
  9. 2018Neural Network Acceptability Judgmentscited by 952
  10. 2019DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphscited by 883
  11. 2023ChatGPT Outperforms Crowd-Workers for Text-Annotation Taskscited by 849
  12. 2018Personalizing Dialogue Agents: I have a dog, do you have pets too?cited by 820
36 more
  1. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 738
  2. 2023Can Large Language Models Be an Alternative to Human Evaluations?cited by 729
  3. 2020SummEval: Re-evaluating Summarization Evaluationcited by 666
  4. 2024OLMo: Accelerating the Science of Language Modelscited by 599
  5. 2023LLM-Blender: Ensembling Large Language Models with Pairwise Ranking and Generative Fusioncited by 550
  6. 2023Zephyr: Direct Distillation of LM Alignmentcited by 498
  7. 2018e-SNLI: Natural Language Inference with Natural Language Explanationscited by 468
  8. 2024Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Modelscited by 431
  9. 2023Is ChatGPT a Good NLG Evaluator? A Preliminary Studycited by 420
  10. 2023Large Language Models Are State-of-the-Art Evaluators of Translation Qualitycited by 416
  11. 2020Asking and Answering Questions to Evaluate the Factual Consistency of Summariescited by 376
  12. 2021Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translationcited by 333
  13. 2024Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Modelscited by 301
  14. 2018Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategiescited by 292
  15. 2023Evaluating Large Language Models at Evaluating Instruction Followingcited by 290
  16. 2019Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoningcited by 281
  17. 2023ToxicChat: Unveiling Hidden Challenges of Toxicity Detection in Real-World User-AI Conversationcited by 224
  18. 2024Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMscited by 224
  19. 2020USR: An Unsupervised and Reference Free Evaluation Metric for Dialog Generationcited by 186
  20. 2022ROSCOE: A Suite of Metrics for Scoring Step-by-Step Reasoningcited by 175
  21. 2023Sociotechnical Safety Evaluation of Generative AI Systemscited by 154
  22. 2024Aligning with Human Judgement: The Role of Pairwise Preference in Large Language Model Evaluatorscited by 146
  23. 2024Large Language Models are Inconsistent and Biased Evaluatorscited by 146
  24. 2023GEMBA-MQM: Detecting Translation Quality Error Spans with GPT-4cited by 143
  25. 2023Topical-Chat: Towards Knowledge-Grounded Open-Domain Conversationscited by 88
  26. 2023LLMs as Narcissistic Evaluators: When Ego Inflates Evaluation Scorescited by 88
  27. 2023Are Large Language Model-based Evaluators the Solution to Scaling Up Multilingual Evaluation?cited by 87
  28. 2023DICES Dataset: Diversity in Conversational AI Evaluation for Safetycited by 86
  29. 2023Exploring the Use of Large Language Models for Reference-Free Text Quality Evaluation: An Empirical Studycited by 85
  30. 2023LLM Comparative Assessment: Zero-shot NLG Evaluation through Pairwise Comparisons using Large Language Modelscited by 66
  31. 2024The Effectiveness of LLMs as Annotators: A Comparative Overview and Empirical Analysis of Direct Representationcited by 29
  32. 2024ChatGPT Rates Natural Language Explanation Quality Like Humans: But on Which Scales?cited by 17
  33. 2024Large Language Models as Financial Data Annotators: A Study on Effectiveness and Efficiencycited by 17
  34. 2024Comparing Inferential Strategies of Humans and Large Language Models in Deductive Reasoningcited by 13
  35. 2022Risk-graded Safety for Handling Medical Queries in Conversational AIcited by 11
  36. 2024PairEval: Open-domain Dialogue Evaluation with Pairwise Comparisoncited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 27; 2025: 103; 2026: 106.

236 papers.

Sort:

By area:

  1. 2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 451
  2. 2024Agent-as-a-Judge: Evaluate Agents with Agentscited by 223
  3. 2025Process Reward Models That Thinkcited by 118
  4. 2025Establishing Best Practices for Building Rigorous Agentic Benchmarkscited by 94
  5. 2025The Alternative Annotator Test for LLM-as-a-Judge: How to Statistically Justify Replacing Human Annotators with LLMscited by 72
  6. 2024Self-Taught Evaluatorscited by 70
  7. 2024Self-Generated Critiques Boost Reward Modeling for Language Modelscited by 69
  8. 2026Rethinking Rubric Generation for Improving LLM Judge and Reward Modeling for Open-ended Taskscited by 55
  9. 2024Limits to scalable evaluation at the frontier: LLM as Judge won't beat twice the datacited by 46
  10. 2024Can LLMs Replace Manual Annotation of Software Engineering Artifacts?cited by 40
  11. 2025A Survey of Test-Time Compute: From Intuitive Inference to Deliberate Reasoningcited by 40
  12. 2025Neither Valid nor Reliable? Investigating the Use of LLMs as Judgescited by 37
38 more
  1. 2025Rating Roulette: Self-Inconsistency in LLM-As-A-Judge Frameworkscited by 37
  2. 2024Style Outweighs Substance: Failure Modes of LLM Judges in Alignment Benchmarkingcited by 35
  3. 2024Evaluating the Evaluator: Measuring LLMs' Adherence to Task Evaluation Instructionscited by 34
  4. 2026DeepResearch Bench II: Diagnosing Deep Research Agents via Rubrics from Expert Reportscited by 33
  5. 2024Beyond correlation: The Impact of Human Uncertainty in Measuring the Effectiveness of Automatic Evaluation and LLM-as-a-Judgecited by 29
  6. 2024JuStRank: Benchmarking LLM Judges for System Rankingcited by 27
  7. 2025Judge's Verdict: A Comprehensive Analysis of LLM Judge Capability Through Human Agreementcited by 24
  8. 2024From Calculation to Adjudication: Examining LLM judges on Mathematical Reasoning Taskscited by 22
  9. 2024Evaluating Creative Short Story Generation in Humans and Large Language Modelscited by 21
  10. 2025Revisiting Uncertainty Quantification Evaluation in Language Models: Spurious Interactions with Response Length Bias Resultscited by 21
  11. 2024Are LLMs Better than Reported? Detecting Label Errors and Mitigating Their Effect on Model Performancecited by 20
  12. 2026Reliability without Validity: A Systematic, Large-Scale Evaluation of LLM-as-a-Judge Models Across Agreement, Consistency, and Biascited by 19
  13. 2024Black-box Uncertainty Quantification Method for LLM-as-a-Judgecited by 18
  14. 2025Hallucination Detection in LLMs with Topological Divergence on Attention Graphscited by 18
  15. 2025Don't Judge Code by Its Cover: Exploring Biases in LLM Judges for Code Evaluationcited by 18
  16. 2025Investigating the Impact of Quantization Methods on the Safety and Reliability of Large Language Modelscited by 17
  17. 2025CondAmbigQA: A Benchmark and Dataset for Conditional Ambiguous Question Answeringcited by 15
  18. 2025GreekBarBench: A Challenging Benchmark for Free-Text Legal Reasoning and Citationscited by 15
  19. 2026Model Context Protocol (MCP) Tool Descriptions Are Smelly! Towards Improving AI Agent Efficiency with Augmented MCP Tool Descriptionscited by 15
  20. 2024GLIDER: Grading LLM Interactions and Decisions using Explainable Rankingcited by 14
  21. 2025Anthropomimetic Uncertainty: What Verbalized Uncertainty in Language Models is Missingcited by 14
  22. 2025Beyond the Surface: Enhancing LLM-as-a-Judge Alignment with Human via Internal Representationscited by 14
  23. 2025E-valuator: Reliable Agent Verifiers with Sequential Hypothesis Testingcited by 14
  24. 2026Diagnosing the Reliability of LLM-as-a-Judge via Item Response Theorycited by 13
  25. 2024On Limitations of LLM as Annotator for Low Resource Languagescited by 12
  26. 2025LLMs can be easily Confused by Instructional Distractionscited by 12
  27. 2025Evaluating the Quality of Benchmark Datasets for Low-Resource Languages: A Case Study on Turkishcited by 12
  28. 2026BiasScope: Towards Automated Detection of Bias in LLM-as-a-Judge Evaluationcited by 12
  29. 2025Forget What You Know about LLMs Evaluations -- LLMs are Like a Chameleoncited by 11
  30. 2025ROC-n-reroll: How verifier imperfection affects test-time scalingcited by 11
  31. 2025Whose View of Safety? A Deep DIVE Dataset for Pluralistic Alignment of Text-to-Image Modelscited by 11
  32. 2024FactLens: Benchmarking Fine-Grained Fact Verificationcited by 10
  33. 2025Has Machine Translation Evaluation Achieved Human Parity? The Human Reference and the Limits of Progresscited by 10
  34. 2025SPHERE: An Evaluation Card for Human-AI Systemscited by 9
  35. 2025Can External Validation Tools Improve Annotation Quality for LLM-as-a-Judge?cited by 9
  36. 2025LeMAJ (Legal LLM-as-a-Judge): Bridging Legal Reasoning and LLM Evaluationcited by 9
  37. 2026Auditing Automated Evaluation, Error Propagation, and Runtime Mitigation in Tool-Using Language Agentscited by 9
  38. 2025Estimating LLM Consistency: A User Baseline vs Surrogate Metricscited by 8

50 most cited of 236 All 236 (JSON)