# Evaluating the Evaluator: Measuring LLMs' Adherence to Task Evaluation Instructions

arXiv: https://arxiv.org/abs/2408.08781 · HTML: https://mlsanity.com/paper/2408.08781/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 34 references that are in the map.
- First posted on arXiv 16 Aug 2024.

## Builds on (most cited first, 26 of 26)

- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [LLM Evaluators Recognize and Favor Their Own Generations](https://mlsanity.com/paper/2404.13076/) (2024)
- [SummEval: Re-evaluating Summarization Evaluation](https://mlsanity.com/paper/2007.12626/) (2020)
- [Prometheus: Inducing Fine-grained Evaluation Capability in Language Models](https://mlsanity.com/paper/2310.08491/) (2023)
- [Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Models](https://mlsanity.com/paper/2405.01535/) (2024)
- [Large Language Models Are State-of-the-Art Evaluators of Translation Quality](https://mlsanity.com/paper/2302.14520/) (2023)
- [LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Tasks](https://mlsanity.com/paper/2406.18403/) (2024)
- [FLASK: Fine-grained Language Model Evaluation based on Alignment Skill Sets](https://mlsanity.com/paper/2307.10928/) (2023)
- [ROSCOE: A Suite of Metrics for Scoring Step-by-Step Reasoning](https://mlsanity.com/paper/2212.07919/) (2022)
- [Generative Judge for Evaluating Alignment](https://mlsanity.com/paper/2310.05470/) (2023)
- [Large Language Models are Inconsistent and Biased Evaluators](https://mlsanity.com/paper/2405.01724/) (2024)
- [Aligning with Human Judgement: The Role of Pairwise Preference in Large Language Model Evaluators](https://mlsanity.com/paper/2403.16950/) (2024)
- [LLM Critics Help Catch LLM Bugs](https://mlsanity.com/paper/2407.00215/) (2024)
- [Shepherd: A Critic for Language Model Generation](https://mlsanity.com/paper/2308.04592/) (2023)
- [Topical-Chat: Towards Knowledge-Grounded Open-Domain Conversations](https://mlsanity.com/paper/2308.11995/) (2023)
- [CriticBench: Benchmarking LLMs for Critique-Correct Reasoning](https://mlsanity.com/paper/2402.14809/) (2024)
- [Exploring the Use of Large Language Models for Reference-Free Text Quality Evaluation: An Empirical Study](https://mlsanity.com/paper/2304.00723/) (2023)
- [Perplexed by Perplexity: Perplexity-Based Data Pruning With Small Reference Models](https://mlsanity.com/paper/2405.20541/) (2024)
- [Benchmarking Generation and Evaluation Capabilities of Large Language Models for Instruction Controllable Summarization](https://mlsanity.com/paper/2311.09184/) (2023)
- [The Next Chapter: A Study of Large Language Models in Storytelling](https://mlsanity.com/paper/2301.09790/) (2023)
- [Finding Blind Spots in Evaluator LLMs with Interpretable Checklists](https://mlsanity.com/paper/2406.13439/) (2024)
- [Exploring ChatGPT's Ability to Rank Content: A Preliminary Study on Consistency with Human Preferences](https://mlsanity.com/paper/2303.07610/) (2023)
- [One Prompt To Rule Them All: LLMs for Opinion Summary Evaluation](https://mlsanity.com/paper/2402.11683/) (2024)
- [OpinSummEval: Revisiting Automated Evaluation for Opinion Summarization](https://mlsanity.com/paper/2310.18122/) (2023)

## Built on by (most cited first, 30 of 34)

- [From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge](https://mlsanity.com/paper/2411.16594/) (2024)
- [RocketEval: Efficient Automated LLM Evaluation via Grading Checklist](https://mlsanity.com/paper/2503.05142/) (2025)
- [CondAmbigQA: A Benchmark and Dataset for Conditional Ambiguous Question Answering](https://mlsanity.com/paper/2502.01523/) (2025)
- [Beyond the Surface: Enhancing LLM-as-a-Judge Alignment with Human via Internal Representations](https://mlsanity.com/paper/2508.03550/) (2025)
- [Beyond the Comfort Zone: Emerging Solutions to Overcome Challenges in Integrating LLMs into Software Products](https://mlsanity.com/paper/2410.12071/) (2024)
- [PersonaEval: Are LLM Evaluators Human Enough to Judge Role-Play?](https://mlsanity.com/paper/2508.10014/) (2025)
- [HELIOS: Adaptive Model And Early-Exit Selection for Efficient LLM Inference Serving](https://mlsanity.com/paper/2504.10724/) (2025)
- [Multi-Agent LLM Judge: automatic personalized LLM judge design for evaluating natural language generation applications](https://mlsanity.com/paper/2504.02867/) (2025)
- [Training Language Model to Critique for Better Refinement](https://mlsanity.com/paper/2506.22157/) (2025)
- [Do LLMs Agree on the Creativity Evaluation of Alternative Uses?](https://mlsanity.com/paper/2411.15560/) (2024)
- [PairBench: Are Vision-Language Models Reliable at Comparing What They See?](https://mlsanity.com/paper/2502.15210/) (2025)
- [Reasoning or Not? A Comprehensive Evaluation of Reasoning LLMs for Dialogue Summarization](https://mlsanity.com/paper/2507.02145/) (2025)
- [Measuring AI "Slop" in Text](https://mlsanity.com/paper/2509.19163/) (2025)
- [HPSS: Heuristic Prompting Strategy Search for LLM Evaluators](https://mlsanity.com/paper/2502.13031/) (2025)
- [ReThinker: Scientific Reasoning by Rethinking with Guided Reflection and Confidence Control](https://mlsanity.com/paper/2602.04496/) (2026)
- [Synthetic Clarification and Correction Dialogues about Data-Centric Tasks -- A Teacher-Student Approach](https://mlsanity.com/paper/2503.14167/) (2025)
- [Large Language Models for Predictive Analysis: How Far Are They?](https://mlsanity.com/paper/2505.17149/) (2025)
- [Safety is Contextual, LLM-Judges Are Not: Navigating the Rigid Priors of Evaluators](https://mlsanity.com/paper/2606.07874/) (2026)
- [SkillVerse : Assessing and Enhancing LLMs with Tree Evaluation](https://mlsanity.com/paper/2506.00319/) (2025)
- [Reward Models are Metrics in a Trench Coat](https://mlsanity.com/paper/2510.03231/) (2025)
- [Believe Your Model: Distribution-Guided Confidence Calibration](https://mlsanity.com/paper/2603.03872/) (2026)
- [Understanding LLM Evaluator Behavior: A Structured Multi-Evaluator Framework for Merchant Risk Assessment](https://mlsanity.com/paper/2602.05110/) (2026)
- [EvalSense: A Framework for Domain-Specific LLM (Meta-)Evaluation](https://mlsanity.com/paper/2602.18823/) (2026)
- [Meta-Evaluating Local LLMs: Rethinking Performance Metrics for Serious Games](https://mlsanity.com/paper/2504.12333/) (2025)
- [Talking to Data: Designing Smart Assistants for Humanities Databases](https://mlsanity.com/paper/2506.00986/) (2025)
- [MTQ-Eval: Multilingual Text Quality Evaluation for Language Models](https://mlsanity.com/paper/2511.09374/) (2025)
- [Evaluating Reward Model Generalization via Pairwise Maximum Discrepancy Competitions](https://mlsanity.com/paper/2601.16987/) (2026)
- [From Binary Groundedness to Support Relations: Towards a Reader-Centred Taxonomy for Comprehension of AI Output](https://mlsanity.com/paper/2604.08082/) (2026)
- [Redefining Quality Criteria and Distance-Aware Score Modeling for Image Editing Assessment](https://mlsanity.com/paper/2604.12175/) (2026)
- [HistoRAG: Embedding Historical Methodology in Retrieval-Augmented Generation Through Critical Technical Practice](https://mlsanity.com/paper/2606.18103/) (2026)

## Abstract (arXiv)

LLMs-as-a-judge is a recently popularized method which replaces human judgements in task evaluation (Zheng et al. 2024) with automatic evaluation using LLMs. Due to widespread use of RLHF (Reinforcement Learning from Human Feedback), state-of-the-art LLMs like GPT4 and Llama3 are expected to have strong alignment with human preferences when prompted for a quality judgement, such as the coherence of a text. While this seems beneficial, it is not clear whether the assessments by an LLM-as-a-judge constitute only an evaluation based on the instructions in the prompts, or reflect its preference for high-quality data similar to its fine-tune data. To investigate how much influence prompting the LLMs-as-a-judge has on the alignment of AI judgements to human judgements, we analyze prompts with increasing levels of instructions about the target quality of an evaluation, for several LLMs-as-a-judge. Further, we compare to a prompt-free method using model perplexity as a quality measure instead. We aggregate a taxonomy of quality criteria commonly used across state-of-the-art evaluations with LLMs and provide this as a rigorous benchmark of models as judges. Overall, we show that the LLMs-as-a-judge benefit only little from highly detailed instructions in prompts and that perplexity can sometimes align better with human judgements than prompting, especially on textual quality.