mlsanityData to 7 Oct 2026

JudgeBench: A Benchmark for Evaluating LLM-based Judges

arXiv 2410.12784 ↗First posted 16 Oct 2024cs.AI cs.CL cs.LGcited by 294

Not on the map. Why?

Cited by 56 in LLM alignment / RLHF / preference optimisation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

LLM-based judges have emerged as a scalable alternative to human evaluation and are increasingly used to assess, compare, and improve models. However, the reliability of LLM-based judges themselves is rarely scrutinized. As LLMs become more advanced, their responses grow more sophisticated, requiring stronger judges to evaluate them. Existing benchmarks primarily focus on a judge's alignment with human preferences, but often fail to account for more challenging tasks where crowdsourced human preference is a poor indicator of factual and logical correctness. To address this, we propose a novel evaluation framework to objectively evaluate LLM-based judges. Based on this framework, we propose JudgeBench, a benchmark for evaluating LLM-based judges on challenging response pairs spanning knowledge, reasoning, math, and coding. JudgeBench leverages a novel pipeline for converting existing difficult datasets into challenging response pairs with preference labels reflecting objective correctness. Our comprehensive evaluation on a collection of prompted judges, fine-tuned judges, multi-agent judges, and reward models shows that JudgeBench poses a significantly greater challenge than previous benchmarks, with many strong models (e.g., GPT-4o) performing just slightly better than random guessing. Overall, JudgeBench offers a reliable platform for assessing increasingly advanced LLM-based judges. Data and code are available at https://github.com/ScalerLab/JudgeBench.

Key facts. Cited by 294 papers in the map (data through 7 Oct 2026). Builds on 47 of its 60 references that are in the map. First posted on arXiv 16 Oct 2024.

Builds on

47 of 60 references.

References · 47

  1. 2024The Llama 3 Herd of Modelscited by 14,192
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  5. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  6. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  7. 2017Deep reinforcement learning from human preferencescited by 3,846
  8. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,634
  9. 2023Let's Verify Step by Stepcited by 3,598
  10. 2023Mistral 7Bcited by 3,065
  11. 2020Learning to summarize from human feedbackcited by 2,562
  12. 2019Fine-Tuning Language Models from Human Preferencescited by 2,124
35 more
  1. 2024LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Codecited by 1,906
  2. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,767
  3. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,758
  4. 2023Large Language Models are not Fair Evaluatorscited by 1,050
  5. 2021A General Language Assistant as a Laboratory for Alignmentcited by 1,023
  6. 2024Large Language Monkeys: Scaling Inference Compute with Repeated Samplingcited by 946
  7. 2022Solving math word problems with process- and outcome-based feedbackcited by 871
  8. 2023AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedbackcited by 738
  9. 2023ChatEval: Towards Better LLM-based Evaluators through Multi-Agent Debatecited by 724
  10. 2024Self-Rewarding Language Modelscited by 654
  11. 2023Prometheus: Inducing Fine-grained Evaluation Capability in Language Modelscited by 558
  12. 2024From Crowdsourced Data to High-Quality Benchmarks: Arena-Hard and BenchBuilder Pipelinecited by 484
  13. 2024RewardBench: Evaluating Reward Models for Language Modelingcited by 480
  14. 2023XSTest: A Test Suite for Identifying Exaggerated Safety Behaviours in Large Language Modelscited by 477
  15. 2024Generative Verifiers: Reward Modeling as Next-Token Predictioncited by 465
  16. 2024WildGuard: Open One-Stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMscited by 460
  17. 2024Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Modelscited by 431
  18. 2024Improve Mathematical Reasoning in Language Models by Automated Process Supervisioncited by 426
  19. 2022Self-critiquing models for assisting human evaluatorscited by 372
  20. 2023JudgeLM: Fine-tuned Large Language Models are Scalable Judgescited by 351
  21. 2023PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimizationcited by 311
  22. 2024Replacing Judges with Juries: Evaluating LLM Generations with a Panel of Diverse Modelscited by 301
  23. 2023Evaluating Large Language Models at Evaluating Instruction Followingcited by 290
  24. 2021Understanding Dataset Difficulty with V-Usable Informationcited by 288
  25. 2024HelpSteer2: Open-source dataset for training top-performing reward modelscited by 232
  26. 2023OctoPack: Instruction Tuning Code Large Language Modelscited by 178
  27. 2023Do-Not-Answer: A Dataset for Evaluating Safeguards in LLMscited by 176
  28. 2023Generative Judge for Evaluating Alignmentcited by 163
  29. 2024Regularizing Hidden States Enables Learning Generalizable Reward Model for LLMscited by 142
  30. 2023Wider and Deeper LLM Networks are Fairer LLM Evaluatorscited by 91
  31. 2024The BiGGen Bench: A Principled Benchmark for Fine-grained Evaluation of Language Models with Language Modelscited by 89
  32. 2023OVM, Outcome-supervised Value Models for Planning in Mathematical Reasoningcited by 84
  33. 2024Systematic Evaluation of LLM-as-a-Judge in LLM Alignment Tasks: Explainable Metrics and Diverse Prompt Templatescited by 71
  34. 2024Explaining Length Bias in LLM-Based Preference Evaluationscited by 48
  35. 2024Arena Learning: Build Data Flywheel for LLMs Post-training via Simulated Chatbot Arenacited by 26

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 106; 2026: 183.

294 papers.

Sort:

By area:

  1. 2024From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judgecited by 480
  2. 2024LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methodscited by 451
  3. 2025Towards Reasoning Era: A Survey of Long Chain-of-Thought for Reasoning Large Language Modelscited by 381
  4. 2025From System 1 to System 2: A Survey of Reasoning Large Language Modelscited by 319
  5. 2025A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?cited by 215
  6. 2025Skywork-Reward-V2: Scaling Preference Data Curation via Human-AI Synergycited by 204
  7. 2025Which Agent Causes Task Failures and When? On Automated Failure Attribution of LLM Multi-Agent Systemscited by 170
  8. 2025Llama-Nemotron: Efficient Reasoning Modelscited by 151
  9. 2025From LLM Reasoning to Autonomous AI Agents: A Comprehensive Reviewcited by 146
  10. 2025A Survey of Frontiers in LLM Reasoning: Inference Scaling, Learning to Reason, and Agentic Systemscited by 108
  11. 2024VL-RewardBench: A Challenging Benchmark for Vision-Language Generative Reward Modelscited by 85
  12. 2025Nemotron 3 Nano: Open, Efficient Mixture-of-Experts Hybrid Mamba-Transformer Model for Agentic Reasoningcited by 85
38 more
  1. 2025ReMA: Learning to Meta-think for LLMs with Multi-Agent Reinforcement Learningcited by 78
  2. 2025JudgeLRM: Large Reasoning Models as a Judgecited by 76
  3. 2025Learning to Plan & Reason for Evaluation with Thinking-LLM-as-a-Judgecited by 75
  4. 2025J1: Incentivizing Thinking in LLM-as-a-Judge via Reinforcement Learningcited by 73
  5. 2025The Alternative Annotator Test for LLM-as-a-Judge: How to Statistically Justify Replacing Human Annotators with LLMscited by 72
  6. 2025HelpSteer3-Preference: Open Human-Annotated Preference Data across Diverse Tasks and Languagescited by 71
  7. 2026Rethinking Rubric Generation for Improving LLM Judge and Reward Modeling for Open-ended Taskscited by 55
  8. 2025No Free Labels: Limitations of LLM-as-a-Judge Without Human Groundingcited by 50
  9. 2025Agentic Reward Modeling: Integrating Human Preferences with Verifiable Correctness Signals for Reliable Reward Systemscited by 49
  10. 2025Evaluating Judges as Evaluators: The JETTS Benchmark of LLM-as-Judges as Test-Time Scaling Evaluatorscited by 42
  11. 2025Multimodal RewardBench: Holistic Evaluation of Reward Models for Vision Language Modelscited by 40
  12. 2025Teaching Language Models to Critique via Reinforcement Learningcited by 38
  13. 2025Answer Matching Outperforms Multiple Choice for Language Model Evaluationcited by 37
  14. 2025Rating Roulette: Self-Inconsistency in LLM-As-A-Judge Frameworkscited by 37
  15. 2025Improving LLM-as-a-Judge Inference with the Judgment Distributioncited by 36
  16. 2025X-MAS: Towards Building Multi-Agent Systems with Heterogeneous LLMscited by 33
  17. 2025CodeJudgeBench: Benchmarking LLM-as-a-Judge for Coding Taskscited by 32
  18. 2025Assessing Judging Bias in Large Reasoning Models: An Empirical Studycited by 30
  19. 2025Auto-Rubric: Learning From Implicit Weights to Explicit Rubrics for Reward Modelingcited by 29
  20. 2024JuStRank: Benchmarking LLM Judges for System Rankingcited by 27
  21. 2026Open Rubric System: Scaling Reinforcement Learning with Pairwise Adaptive Rubriccited by 26
  22. 2025Judge's Verdict: A Comprehensive Analysis of LLM Judge Capability Through Human Agreementcited by 24
  23. 2025Does Context Matter? ContextualJudgeBench for Evaluating LLM-based Judges in Contextual Settingscited by 22
  24. 2025TrustJudge: Inconsistencies of LLM-as-a-Judge and How to Alleviate Themcited by 20
  25. 2025Multi-Agent Debate for LLM Judges with Adaptive Stability Detectioncited by 20
  26. 2025Hard2Verify: A Step-Level Verification Benchmark for Open-Ended Frontier Mathcited by 19
  27. 2026RubricEval: A Rubric-Level Meta-Evaluation Benchmark for LLM Judges in Instruction Followingcited by 19
  28. 2026Reliability without Validity: A Systematic, Large-Scale Evaluation of LLM-as-a-Judge Models Across Agreement, Consistency, and Biascited by 19
  29. 2025Don't Judge Code by Its Cover: Exploring Biases in LLM Judges for Code Evaluationcited by 18
  30. 2025Variation in Verification: Understanding Verification Dynamics in Large Language Modelscited by 18
  31. 2025Overconfidence in LLM-as-a-Judge: Diagnosis and Confidence-Driven Solutioncited by 17
  32. 2025Leveraging LLMs as Meta-Judges: A Multi-Agent Framework for Evaluating LLM Judgmentscited by 16
  33. 2025When AIs Judge AIs: The Rise of Agent-as-a-Judge Evaluation for LLMscited by 16
  34. 2025Incentivizing Agentic Reasoning in LLM Judges via Tool-Integrated Reinforcement Learningcited by 16
  35. 2025Multimodal RewardBench 2: Evaluating Omni Reward Models for Interleaved Text and Imagecited by 16
  36. 2025CodeCriticBench: A Holistic Code Critique Benchmark for Large Language Modelscited by 14
  37. 2025Personalized Safety in LLMs: A Benchmark and A Planning-Based Agent Approachcited by 14
  38. 2025Reliable Fine-Grained Evaluation of Natural Language Math Proofscited by 14

50 most cited of 294 All 294 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Are We on the Right Way to Assessing LLM-as-a-Judge?
  2. 2025Verdict: A Library for Scaling Judge-Time Compute
  3. 2024From Calculation to Adjudication: Examining LLM judges on Mathematical Reasoning Tasks
  4. 2024Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judges
  5. 2026Quantifying and Mitigating Self-Preference Bias of LLM Judges
  6. 2025Who Judges the Judge? LLM Jury-on-Demand: Building Trustworthy LLM Evaluation Systems
  7. 2026JudgeStealer: Extracting LLM Judging Capabilities across Evaluation Protocols
  8. 2025Explicit Reasoning Makes Better Judges: A Systematic Study on Accuracy, Efficiency, and Robustness