# CLEAR: Error Analysis via LLM-as-a-Judge Made Easy

arXiv: https://arxiv.org/abs/2507.18392 · HTML: https://mlsanity.com/paper/2507.18392/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 22 references that are in the map.
- First posted on arXiv 24 Jul 2025.

## Builds on (most cited first, 15 of 15)

- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs](https://mlsanity.com/paper/2307.16789/) (2023)
- [Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preference](https://mlsanity.com/paper/2403.04132/) (2024)
- [RAGBench: Explainable Benchmark for Retrieval-Augmented Generation Systems](https://mlsanity.com/paper/2407.11005/) (2024)
- [From Arguments to Key Points: Towards Automatic Argument Summarization](https://mlsanity.com/paper/2005.01619/) (2020)
- [DelucionQA: Detecting Hallucinations in Domain-specific Question Answering](https://mlsanity.com/paper/2312.05200/) (2023)
- [Quantitative Argument Summarization and Beyond: Cross-Domain Key Point Analysis](https://mlsanity.com/paper/2010.05369/) (2020)
- [Genie: Achieving Human Parity in Content-Grounded Datasets Generation](https://mlsanity.com/paper/2401.14367/) (2024)
- [JuStRank: Benchmarking LLM Judges for System Ranking](https://mlsanity.com/paper/2412.09569/) (2024)
- [EvalTree: Profiling Language Model Weaknesses via Hierarchical Capability Trees](https://mlsanity.com/paper/2503.08893/) (2025)
- [QualEval: Qualitative Evaluation for Model Improvement](https://mlsanity.com/paper/2311.02807/) (2023)

## Built on by (most cited first, 3 of 3)

- [Doctor-RAG: A Failure-Aware Repair Framework for Agentic Retrieval-Augmented Generation](https://mlsanity.com/paper/2604.00865/) (2026)
- [AgentCoMa: A Compositional Benchmark Mixing Commonsense and Mathematical Reasoning in Real-World Scenarios](https://mlsanity.com/paper/2508.19988/) (2025)
- [Agentic CLEAR: Automating Multi-Level Evaluation of LLM Agents](https://mlsanity.com/paper/2605.22608/) (2026)

## Abstract (arXiv)

The evaluation of Large Language Models (LLMs) increasingly relies on other LLMs acting as judges. However, current evaluation paradigms typically yield a single score or ranking, answering which model is better but not why. While essential for benchmarking, these top-level scores obscure the specific, actionable reasons behind a model's performance. To bridge this gap, we introduce CLEAR, an interactive, open-source package for LLM-based error analysis. CLEAR first generates per-instance textual feedback, then it creates a set of system-level error issues, and quantifies the prevalence of each identified issue. Our package also provides users with an interactive dashboard that allows for a comprehensive error analysis through aggregate visualizations, applies interactive filters to isolate specific issues or score ranges, and drills down to the individual instances that exemplify a particular behavioral pattern. We demonstrate CLEAR analysis for RAG and Math benchmarks, and showcase its utility through a user case study.