# LLMEval: A Preliminary Study on How to Evaluate Large Language Models

arXiv: https://arxiv.org/abs/2312.07398 · HTML: https://mlsanity.com/paper/2312.07398/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 14 references that are in the map.
- First posted on arXiv 12 Dec 2023.

## Builds on (most cited first, 10 of 10)

- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [Large Language Models are not Fair Evaluators](https://mlsanity.com/paper/2305.17926/) (2023)
- [AlpacaFarm: A Simulation Framework for Methods that Learn from Human Feedback](https://mlsanity.com/paper/2305.14387/) (2023)
- [C-Eval: A Multi-Level Multi-Discipline Chinese Evaluation Suite for Foundation Models](https://mlsanity.com/paper/2305.08322/) (2023)
- [Is ChatGPT a Good NLG Evaluator? A Preliminary Study](https://mlsanity.com/paper/2303.04048/) (2023)
- [MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distance](https://mlsanity.com/paper/1909.02622/) (2019)
- [GPTScore: Evaluate as You Desire](https://mlsanity.com/paper/2302.04166/) (2023)

## Built on by (most cited first, 23 of 23)

- [Judging the Judges: Evaluating Alignment and Vulnerabilities in LLMs-as-Judges](https://mlsanity.com/paper/2406.12624/) (2024)
- [Fine-tuning large language models for domain adaptation: Exploration of training strategies, scaling, model merging and synergistic capabilities](https://mlsanity.com/paper/2409.03444/) (2024)
- [Inherent Trade-Offs between Diversity and Stability in Multi-Task Benchmarks](https://mlsanity.com/paper/2405.01719/) (2024)
- [Beyond Task Completion: Revealing Corrupt Success in LLM Agents through Procedure-Aware Evaluation](https://mlsanity.com/paper/2603.03116/) (2026)
- [From GPT-4 to Gemini and Beyond: Assessing the Landscape of MLLMs on Generalizability, Trustworthiness and Causality through Four Modalities](https://mlsanity.com/paper/2401.15071/) (2024)
- [Exploring the Trade-Offs: Quantization Methods, Task Difficulty, and Model Size in Large Language Models From Edge to Giant](https://mlsanity.com/paper/2409.11055/) (2024)
- [MLaKE: Multilingual Knowledge Editing Benchmark for Large Language Models](https://mlsanity.com/paper/2404.04990/) (2024)
- [am-ELO: A Stable Framework for Arena-based LLM Evaluation](https://mlsanity.com/paper/2505.03475/) (2025)
- [RAGRouter: Learning to Route Queries to Multiple Retrieval-Augmented Language Models](https://mlsanity.com/paper/2505.23052/) (2025)
- [Quantitative LLM Judges](https://mlsanity.com/paper/2506.02945/) (2025)
- [LLM-as-Judge on a Budget](https://mlsanity.com/paper/2602.15481/) (2026)
- [ArXivBench: When You Should Avoid Using ChatGPT for Academic Writing](https://mlsanity.com/paper/2504.10496/) (2025)
- [A Principle-based Framework for the Development and Evaluation of Large Language Models for Health and Wellness](https://mlsanity.com/paper/2512.08936/) (2025)
- [PR-CAD: Progressive Refinement for Unified Controllable and Faithful Text-to-CAD Generation with Large Language Models](https://mlsanity.com/paper/2604.19773/) (2026)
- [Evaluating LLMs on Entity Disambiguation in Tables](https://mlsanity.com/paper/2408.06423/) (2024)
- [LLMEval-Fair: A Large-Scale Longitudinal Study on Robust and Fair Evaluation of Large Language Models](https://mlsanity.com/paper/2508.05452/) (2025)
- [What Makes an Evaluation Useful? Common Pitfalls and Best Practices](https://mlsanity.com/paper/2503.23424/) (2025)
- [From Reddit to Generative AI: Evaluating Large Language Models for Anxiety Support Fine-tuned on Social Media Data](https://mlsanity.com/paper/2505.18464/) (2025)
- [Key Considerations for Domain Expert Involvement in LLM Design and Evaluation: An Ethnographic Study](https://mlsanity.com/paper/2602.14357/) (2026)
- [A Conceptual Framework for AI Capability Evaluations](https://mlsanity.com/paper/2506.18213/) (2025)
- [A Note on Code Quality Score: LLMs for Maintainable Large Codebases](https://mlsanity.com/paper/2508.02732/) (2025)
- [Generative Artificial Intelligence Extracts Structure-Function Relationships from Plants for New Materials](https://mlsanity.com/paper/2508.06591/) (2025)
- [CiteGuard-RAG: A Validation-Centered AI System for Evidence-Grounded Question Answering](https://mlsanity.com/paper/2609.15830/) (2026)

## Abstract (arXiv)

Recently, the evaluation of Large Language Models has emerged as a popular area of research. The three crucial questions for LLM evaluation are ``what, where, and how to evaluate''. However, the existing research mainly focuses on the first two questions, which are basically what tasks to give the LLM during testing and what kind of knowledge it should deal with. As for the third question, which is about what standards to use, the types of evaluators, how to score, and how to rank, there hasn't been much discussion. In this paper, we analyze evaluation methods by comparing various criteria with both manual and automatic evaluation, utilizing onsite, crowd-sourcing, public annotators and GPT-4, with different scoring methods and ranking systems. We propose a new dataset, LLMEval and conduct evaluations on 20 LLMs. A total of 2,186 individuals participated, leading to the generation of 243,337 manual annotations and 57,511 automatic evaluation results. We perform comparisons and analyses of different settings and conduct 10 conclusions that can provide some insights for evaluating LLM in the future. The dataset and the results are publicly available at https://github.com/llmeval .