# Finding Blind Spots in Evaluator LLMs with Interpretable Checklists

arXiv: https://arxiv.org/abs/2406.13439 · HTML: https://mlsanity.com/paper/2406.13439/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 51 references that are in the map.
- First posted on arXiv 19 Jun 2024.

## Builds on (most cited first, 30 of 34)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context](https://mlsanity.com/paper/2403.05530/) (2024)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://mlsanity.com/paper/2305.14251/) (2023)
- [WizardLM: Empowering large pre-trained language models to follow complex instructions](https://mlsanity.com/paper/2304.12244/) (2023)
- [LIMA: Less Is More for Alignment](https://mlsanity.com/paper/2305.11206/) (2023)
- [Large Language Models are not Fair Evaluators](https://mlsanity.com/paper/2305.17926/) (2023)
- [Beyond Accuracy: Behavioral Testing of NLP models with CheckList](https://mlsanity.com/paper/2005.04118/) (2020)
- [Enhancing Chat Language Models by Scaling High-quality Instructional Conversations](https://mlsanity.com/paper/2305.14233/) (2023)
- [AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback](https://mlsanity.com/paper/2305.14387/) (2023)
- [Can Large Language Models Be an Alternative to Human Evaluations?](https://mlsanity.com/paper/2305.01937/) (2023)
- [LLM Evaluators Recognize and Favor Their Own Generations](https://mlsanity.com/paper/2404.13076/) (2024)
- [Prometheus: Inducing Fine-grained Evaluation Capability in Language Models](https://mlsanity.com/paper/2310.08491/) (2023)
- [Prometheus 2: An Open Source Language Model Specialized in Evaluating Other Language Models](https://mlsanity.com/paper/2405.01535/) (2024)
- [Is ChatGPT a Good NLG Evaluator? A Preliminary Study](https://mlsanity.com/paper/2303.04048/) (2023)
- [Large Language Models Are State-of-the-Art Evaluators of Translation Quality](https://mlsanity.com/paper/2302.14520/) (2023)
- [JudgeLM: Fine-tuned Large Language Models are Scalable Judges](https://mlsanity.com/paper/2310.17631/) (2023)
- [PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimization](https://mlsanity.com/paper/2306.05087/) (2023)
- [Evaluating Large Language Models at Evaluating Instruction Following](https://mlsanity.com/paper/2310.07641/) (2023)
- [Reasoning or Reciting? Exploring the Capabilities and Limitations of Language Models Through Counterfactual Tasks](https://mlsanity.com/paper/2307.02477/) (2023)
- [FLASK: Fine-grained Language Model Evaluation based on Alignment Skill Sets](https://mlsanity.com/paper/2307.10928/) (2023)
- [Tangled up in BLEU: Reevaluating the Evaluation of Automatic Machine Translation Evaluation Metrics](https://mlsanity.com/paper/2006.06264/) (2020)
- [Branch-Solve-Merge Improves Large Language Model Evaluation and Generation](https://mlsanity.com/paper/2310.15123/) (2023)
- [Wider and Deeper LLM Networks are Fairer LLM Evaluators](https://mlsanity.com/paper/2308.01862/) (2023)
- [The BiGGen Bench: A Principled Benchmark for Fine-grained Evaluation of Language Models with Language Models](https://mlsanity.com/paper/2406.05761/) (2024)
- [Evaluating the Instruction-Following Robustness of Large Language Models to Prompt Injection](https://mlsanity.com/paper/2308.10819/) (2023)
- [Perturbation CheckLists for Evaluating NLG Evaluation Metrics](https://mlsanity.com/paper/2109.05771/) (2021)

## Built on by (most cited first, 30 of 32)

- [From Generation to Judgment: Opportunities and Challenges of LLM-as-a-judge](https://mlsanity.com/paper/2411.16594/) (2024)
- [LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods](https://mlsanity.com/paper/2412.05579/) (2024)
- [Evaluating the Evaluator: Measuring LLMs' Adherence to Task Evaluation Instructions](https://mlsanity.com/paper/2408.08781/) (2024)
- [How Reliable is Multilingual LLM-as-a-Judge?](https://mlsanity.com/paper/2505.12201/) (2025)
- [An Empirical Analysis of Uncertainty in Large Language Model Evaluations](https://mlsanity.com/paper/2502.10709/) (2025)
- [An Empirical Study of LLM-as-a-Judge: How Design Choices Impact Evaluation Reliability](https://mlsanity.com/paper/2506.13639/) (2025)
- [Black-box Uncertainty Quantification Method for LLM-as-a-Judge](https://mlsanity.com/paper/2410.11594/) (2024)
- [Cross-Lingual Auto Evaluation for Assessing Multilingual LLMs](https://mlsanity.com/paper/2410.13394/) (2024)
- [Cognitive Overload Attack:Prompt Injection for Long Context](https://mlsanity.com/paper/2410.11272/) (2024)
- [Tuning LLM Judge Design Decisions for 1/1000 of the Cost](https://mlsanity.com/paper/2501.17178/) (2025)
- [Can Vision-Language Models Evaluate Handwritten Math?](https://mlsanity.com/paper/2501.07244/) (2025)
- [LLMs can Perform Multi-Dimensional Analytic Writing Assessments: A Case Study of L2 Graduate-Level Academic English Writing](https://mlsanity.com/paper/2502.11368/) (2025)
- [Judge Reliability Harness: Stress Testing the Reliability of LLM Judges](https://mlsanity.com/paper/2603.05399/) (2026)
- [Awes, Laws, and Flaws From Today's LLM Research](https://mlsanity.com/paper/2408.15409/) (2024)
- [HPSS: Heuristic Prompting Strategy Search for LLM Evaluators](https://mlsanity.com/paper/2502.13031/) (2025)
- [HEALTH-PARIKSHA: Assessing RAG Models for Health Chatbots in Real-World Multilingual Settings](https://mlsanity.com/paper/2410.13671/) (2024)
- [Automated Validation of LLM-based Evaluators for Software Engineering Artifacts](https://mlsanity.com/paper/2508.02827/) (2025)
- [Turning Bias into Bugs: Bandit-Guided Style Manipulation Attacks on LLM Judges](https://mlsanity.com/paper/2605.26156/) (2026)
- [LLM Judges Can Be Too Generous When There Is No Reference Answer](https://mlsanity.com/paper/2607.12885/) (2026)
- [Checklist Engineering Empowers Multilingual LLM Judges](https://mlsanity.com/paper/2507.06774/) (2025)
- [Taming LLMs with Negative Samples: A Reference-Free Framework to Evaluate Presentation Content with Actionable Feedback](https://mlsanity.com/paper/2505.18240/) (2025)
- [Generate, Evaluate, Iterate: Synthetic Data for Human-in-the-Loop Refinement of LLM Judges](https://mlsanity.com/paper/2511.04478/) (2025)
- [trajectory-judge: What Outcome-Only LLM Judges Miss on Agent Trajectories](https://mlsanity.com/paper/2609.00038/) (2026)
- [If Eleanor Rigby Had Met ChatGPT: A Study on Loneliness in a Post-LLM World](https://mlsanity.com/paper/2412.01617/) (2024)
- [Seeing Isn't Believing: Uncovering Blind Spots in Evaluator Vision-Language Models](https://mlsanity.com/paper/2604.21523/) (2026)
- [When Does a Second Model Help? Cross-Model Review in LLM Verification](https://mlsanity.com/paper/2610.01471/) (2026)
- [Cascaded Information Disclosure for Generalized Evaluation of Problem Solving Capabilities](https://mlsanity.com/paper/2507.23776/) (2025)
- [Accuracy Hides How Language Models Fail: Measuring Failure States Under Matched Output Budgets](https://mlsanity.com/paper/2607.24268/) (2026)
- [Can We Trust the Judges? Validation of Factuality Evaluation Methods via Answer Perturbation](https://mlsanity.com/paper/2609.15561/) (2026)
- [PACE: Precise AI Cinematic Expression](https://mlsanity.com/paper/2609.19853/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) are increasingly relied upon to evaluate text outputs of other LLMs, thereby influencing leaderboards and development decisions. However, concerns persist over the accuracy of these assessments and the potential for misleading conclusions. In this work, we investigate the effectiveness of LLMs as evaluators for text generation tasks. We propose FBI, a novel framework designed to examine the proficiency of Evaluator LLMs in assessing four critical abilities in other LLMs: factual accuracy, instruction following, coherence in long-form writing, and reasoning proficiency. By introducing targeted perturbations in answers generated by LLMs, that clearly impact one of these key capabilities, we test whether an Evaluator LLM can detect these quality drops. By creating a total of 2400 perturbed answers covering 22 perturbation categories, we conduct a comprehensive study using different evaluation strategies on five prominent LLMs commonly used as evaluators in the literature. Our findings reveal significant shortcomings in current Evaluator LLMs, which failed to identify quality drops in over 50% of cases on average. Single-answer and pairwise evaluations demonstrated notable limitations, whereas reference-based evaluations showed comparatively better performance. These results underscore the unreliable nature of current Evaluator LLMs and advocate for cautious implementation in practical applications. Code and data are available at https://github.com/AI4Bharat/FBI.