# xFinder: Large Language Models as Automated Evaluators for Reliable Evaluation

arXiv: https://arxiv.org/abs/2405.11874 · HTML: https://mlsanity.com/paper/2405.11874/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 45 references that are in the map.
- First posted on arXiv 20 May 2024.

## Builds on (most cited first, 30 of 32)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge](https://mlsanity.com/paper/1803.05457/) (2018)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://mlsanity.com/paper/2310.11511/) (2023)
- [CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge](https://mlsanity.com/paper/1811.00937/) (2018)
- [Prometheus: Inducing Fine-grained Evaluation Capability in Language Models](https://mlsanity.com/paper/2310.08491/) (2023)
- [JudgeLM: Fine-tuned Large Language Models are Scalable Judges](https://mlsanity.com/paper/2310.17631/) (2023)
- [PandaLM: An Automatic Evaluation Benchmark for LLM Instruction Tuning Optimization](https://mlsanity.com/paper/2306.05087/) (2023)
- [Proving Test Set Contamination in Black Box Language Models](https://mlsanity.com/paper/2310.17623/) (2023)
- [Evaluating Large Language Models: A Comprehensive Survey](https://mlsanity.com/paper/2310.19736/) (2023)
- [Don't Make Your LLM an Evaluation Benchmark Cheater](https://mlsanity.com/paper/2311.01964/) (2023)
- [Generative Judge for Evaluating Alignment](https://mlsanity.com/paper/2310.05470/) (2023)
- [Political Compass or Spinning Arrow? Towards More Meaningful Evaluations for Values and Opinions in Large Language Models](https://mlsanity.com/paper/2402.16786/) (2024)
- [Factuality Challenges in the Era of Large Language Models](https://mlsanity.com/paper/2310.05189/) (2023)
- [Benchmarking LLMs via Uncertainty Quantification](https://mlsanity.com/paper/2401.12794/) (2024)
- ["My Answer is C": First-Token Probabilities Do Not Match Text Answers in Instruction-Tuned Language Models](https://mlsanity.com/paper/2402.14499/) (2024)
- [Evaluating the Zero-shot Robustness of Instruction-tuned Language Models](https://mlsanity.com/paper/2306.11270/) (2023)
- [Can we trust the evaluation on ChatGPT?](https://mlsanity.com/paper/2303.12767/) (2023)
- [CRUD-RAG: A Comprehensive Chinese Benchmark for Retrieval-Augmented Generation of Large Language Models](https://mlsanity.com/paper/2401.17043/) (2024)
- [Improving the Robustness of Large Language Models via Consistency Alignment](https://mlsanity.com/paper/2403.14221/) (2024)
- [Robustness of Learning from Task Instructions](https://mlsanity.com/paper/2212.03813/) (2022)
- [Evaluating Open-QA Evaluation](https://mlsanity.com/paper/2305.12421/) (2023)
- [Are Large Language Models Reliable Judges? A Study on the Factuality Evaluation Capabilities of LLMs](https://mlsanity.com/paper/2311.00681/) (2023)
- [UltraEval: A Lightweight Platform for Flexible and Comprehensive Evaluation for LLMs](https://mlsanity.com/paper/2404.07584/) (2024)

## Built on by (most cited first, 19 of 19)

- [Internal Consistency and Self-Feedback in Large Language Models: A Survey](https://mlsanity.com/paper/2407.14507/) (2024)
- [BMMR: A Large-Scale Bilingual Multimodal Multi-Discipline Reasoning Dataset](https://mlsanity.com/paper/2507.03483/) (2025)
- [Benchmarks Saturate When The Model Gets Smarter Than The Judge](https://mlsanity.com/paper/2601.19532/) (2026)
- [Script Fragmentation and Format: What Drives the English-Bengali Performance Gap in Open LLMs?](https://mlsanity.com/paper/2507.23248/) (2025)
- [TurtleBench: Evaluating Top Language Models via Real-World Yes/No Puzzles](https://mlsanity.com/paper/2410.05262/) (2024)
- [GRAPHMOE: Amplifying Cognitive Depth of Mixture-of-Experts Network via Introducing Self-Rethinking Mechanism](https://mlsanity.com/paper/2501.07890/) (2025)
- [Improving LLM First-Token Predictions in Multiple-Choice Question Answering via Output Prefilling](https://mlsanity.com/paper/2505.15323/) (2025)
- [Reward Model Routing in Alignment](https://mlsanity.com/paper/2510.02850/) (2025)
- [GuessArena: Guess Who I Am? A Self-Adaptive Framework for Evaluating LLMs in Domain-Specific Knowledge and Reasoning](https://mlsanity.com/paper/2505.22661/) (2025)
- [ReTraceQA: Evaluating Reasoning Traces of Small Language Models in Commonsense Question Answering](https://mlsanity.com/paper/2510.09351/) (2025)
- [Beyond Theorem Proving: Formulation, Framework and Benchmark for Formal Problem-Solving](https://mlsanity.com/paper/2505.04528/) (2025)
- [Mind the Gap: Bridging Thought Leap for Improved Chain-of-Thought Tuning](https://mlsanity.com/paper/2505.14684/) (2025)
- [Data Swarms: Optimizable Generation of Synthetic Evaluation Data](https://mlsanity.com/paper/2506.00741/) (2025)
- [GSM-SEM: Benchmark and Framework for Generating Semantically Variant Augmentations](https://mlsanity.com/paper/2605.07053/) (2026)
- [Clustering Algorithms and RAG Enhancing Semi-Supervised Text Classification with Large LLMs](https://mlsanity.com/paper/2411.06175/) (2024)
- [Exploring Fine-Tuning for In-Context Retrieval and Efficient KV-Caching in Long-Context Language Models](https://mlsanity.com/paper/2601.18527/) (2026)
- [Incorporating Token Usage into Prompting Strategy Evaluation](https://mlsanity.com/paper/2505.14880/) (2025)
- [A Function-Level Vulnerability Score Measures Flag Rate More Than the Model: Protocol Effects on Paired Benchmarks](https://mlsanity.com/paper/2609.32890/) (2026)
- [Audit Before You Distill: A Controlled Workflow for Reasoning Transfer into Compact Medical Language Models](https://mlsanity.com/paper/2610.16713/) (2026)

## Abstract (arXiv)

The continuous advancement of large language models (LLMs) has brought increasing attention to the critical issue of developing fair and reliable methods for evaluating their performance. Particularly, the emergence of cheating phenomena, such as test set leakage and prompt format overfitting, poses significant challenges to the reliable evaluation of LLMs. As evaluation frameworks commonly use Regular Expression (RegEx) for answer extraction, models may adjust their responses to fit formats easily handled by RegEx. Nevertheless, the key answer extraction module based on RegEx frequently suffers from extraction errors. Furthermore, recent studies proposing fine-tuned LLMs as judge models for automated evaluation face challenges in terms of generalization ability and fairness. This paper comprehensively analyzes the entire LLM evaluation chain and demonstrates that optimizing the key answer extraction module improves extraction accuracy and enhances evaluation reliability. Our findings suggest that improving the key answer extraction module can lead to higher judgment accuracy and improved evaluation efficiency compared to the judge models. To address these issues, we propose xFinder, a novel evaluator for answer extraction and matching in LLM evaluation. As part of this process, we create a specialized dataset, the Key Answer Finder (KAF) dataset, to ensure effective model training and evaluation. Generalization tests and real-world evaluations show that the smallest xFinder model, with only 500 million parameters, achieves an average extraction accuracy of 93.42%. In contrast, RegEx accuracy in the best evaluation framework is 74.38%. The final judgment accuracy of xFinder reaches 97.61%, outperforming existing evaluation frameworks and judge models.