# Toward Generalizable Evaluation in the LLM Era: A Survey Beyond Benchmarks

arXiv: https://arxiv.org/abs/2504.18838 · HTML: https://mlsanity.com/paper/2504.18838/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 308 of its 368 references that are in the map.
- First posted on arXiv 26 Apr 2025.

## Builds on (most cited first, 30 of 308)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [Training Verifiers to Solve Math Word Problems](https://mlsanity.com/paper/2110.14168/) (2021)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena](https://mlsanity.com/paper/2306.05685/) (2023)
- [Evaluating Large Language Models Trained on Code](https://mlsanity.com/paper/2107.03374/) (2021)
- [Measuring Massive Multitask Language Understanding](https://mlsanity.com/paper/2009.03300/) (2020)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Measuring Mathematical Problem Solving With the MATH Dataset](https://mlsanity.com/paper/2103.03874/) (2021)
- [GLUE: A Multi-Task Benchmark and Analysis Platform for Natural Language Understanding](https://mlsanity.com/paper/1804.07461/) (2018)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [HellaSwag: Can a Machine Really Finish Your Sentence?](https://mlsanity.com/paper/1905.07830/) (2019)
- [Finetuned Language Models Are Zero-Shot Learners](https://mlsanity.com/paper/2109.01652/) (2021)
- [HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering](https://mlsanity.com/paper/1809.09600/) (2018)
- [Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2204.05862/) (2022)
- [OPT: Open Pre-trained Transformer Language Models](https://mlsanity.com/paper/2205.01068/) (2022)
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://mlsanity.com/paper/2310.06770/) (2023)
- [Universal and Transferable Adversarial Attacks on Aligned Language Models](https://mlsanity.com/paper/2307.15043/) (2023)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://mlsanity.com/paper/2109.07958/) (2021)
- [TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension](https://mlsanity.com/paper/1705.03551/) (2017)
- [PIQA: Reasoning about Physical Commonsense in Natural Language](https://mlsanity.com/paper/1911.11641/) (2019)
- [Self-Instruct: Aligning Language Models with Self-Generated Instructions](https://mlsanity.com/paper/2212.10560/) (2022)
- [Do As I Can, Not As I Say: Grounding Language in Robotic Affordances](https://mlsanity.com/paper/2204.01691/) (2022)
- [G-Eval: NLG Evaluation using GPT-4 with Better Human Alignment](https://mlsanity.com/paper/2303.16634/) (2023)
- [MMBench: Is Your Multi-modal Model an All-around Player?](https://mlsanity.com/paper/2307.06281/) (2023)
- [LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code](https://mlsanity.com/paper/2403.07974/) (2024)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://mlsanity.com/paper/2206.04615/) (2022)
- [ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIs](https://mlsanity.com/paper/2307.16789/) (2023)

## Built on by (most cited first, 28 of 28)

- [The Price of a Second Thought: On the Evaluation of Reasoning Efficiency in Large Language Models](https://mlsanity.com/paper/2505.22017/) (2025)
- [Disentangling Language and Culture for Evaluating Multilingual Large Language Models](https://mlsanity.com/paper/2505.24635/) (2025)
- [From LLMs to MLLMs to Agents: A Survey of Emerging Paradigms in Jailbreak Attacks and Defenses within LLM Ecosystem](https://mlsanity.com/paper/2506.15170/) (2025)
- [Brain-Inspired Graph Multi-Agent Systems for LLM Reasoning](https://mlsanity.com/paper/2603.15371/) (2026)
- [Reasoning emerges from constrained inference manifolds in large language models](https://mlsanity.com/paper/2605.08142/) (2026)
- [CoDAE: Adapting Large Language Models for Education via Chain-of-Thought Data Augmentation](https://mlsanity.com/paper/2508.08386/) (2025)
- [Memorization in Large Language Models in Medicine: Prevalence, Characteristics, and Implications](https://mlsanity.com/paper/2509.08604/) (2025)
- [Security in LLM-as-a-Judge: A Comprehensive SoK](https://mlsanity.com/paper/2603.29403/) (2026)
- [LexGenius: An Expert-Level Benchmark for Large Language Models in Legal General Intelligence](https://mlsanity.com/paper/2512.04578/) (2025)
- [GuessArena: Guess Who I Am? A Self-Adaptive Framework for Evaluating LLMs in Domain-Specific Knowledge and Reasoning](https://mlsanity.com/paper/2505.22661/) (2025)
- [Data Doping or True Intelligence? Evaluating the Transferability of Injected Knowledge in LLMs](https://mlsanity.com/paper/2505.17140/) (2025)
- [FinEval-KR: A Financial Domain Evaluation Framework for Large Language Models' Knowledge and Reasoning](https://mlsanity.com/paper/2506.21591/) (2025)
- [Safe for Whom? Rethinking How We Evaluate the Safety of LLMs for Real Users](https://mlsanity.com/paper/2512.10687/) (2025)
- [LLM Benchmark Datasets Should Be Contamination-Resistant](https://mlsanity.com/paper/2605.19999/) (2026)
- [Teach2Eval: An Indirect Evaluation Method for LLM by Judging How It Teaches](https://mlsanity.com/paper/2505.12259/) (2025)
- [Continuous Monitoring of Large-Scale Generative AI via Deterministic Knowledge Graph Structures](https://mlsanity.com/paper/2509.03857/) (2025)
- [MaP: A Unified Framework for Reliable Evaluation of Pre-training Dynamics](https://mlsanity.com/paper/2510.09295/) (2025)
- [If I Could Turn Back Time: Temporal Reframing as a Historical Reasoning Task for LLMs](https://mlsanity.com/paper/2511.04432/) (2025)
- [Key Considerations for Domain Expert Involvement in LLM Design and Evaluation: An Ethnographic Study](https://mlsanity.com/paper/2602.14357/) (2026)
- [Beyond Scores: Diagnostic LLM Evaluation via Fine-Grained Abilities](https://mlsanity.com/paper/2604.12191/) (2026)
- [Hint Tuning: Less Data Makes Better Reasoners](https://mlsanity.com/paper/2605.08665/) (2026)
- [Do They Understand Them? An Updated Evaluation on Nonbinary Pronoun Handling in Large Language Models](https://mlsanity.com/paper/2508.00788/) (2025)
- [Financial Instruction Following Evaluation (FIFE)](https://mlsanity.com/paper/2512.08965/) (2025)
- [Toward IIT-Inspired Consciousness in LLMs: A Reward-Based Learning Framework](https://mlsanity.com/paper/2601.22786/) (2026)
- [Benchmarking Large Language Models for Quebec Insurance: From Closed-Book to Retrieval-Augmented Generation](https://mlsanity.com/paper/2603.07825/) (2026)
- [From Feelings to Metrics: Understanding and Formalizing How Users Vibe-Test LLMs](https://mlsanity.com/paper/2604.14137/) (2026)
- [Ontology-Based Contextual AI Evaluations (OB-CAIE) Methodology](https://mlsanity.com/paper/2610.00529/) (2026)
- [Belief-Trajectory Energy: Measuring the Path to a Prediction](https://mlsanity.com/paper/2610.05114/) (2026)

## Abstract (arXiv)

Large Language Models (LLMs) are advancing at an amazing speed and have become indispensable across academia, industry, and daily applications. To keep pace with the status quo, this survey probes the core challenges that the rise of LLMs poses for evaluation. We identify and analyze two pivotal transitions: (i) from task-specific to capability-based evaluation, which reorganizes benchmarks around core competencies such as knowledge, reasoning, instruction following, multi-modal understanding, and safety; and (ii) from manual to automated evaluation, encompassing dynamic dataset curation and "LLM-as-a-judge" scoring. Yet, even with these transitions, a crucial obstacle persists: the evaluation generalization issue. Bounded test sets cannot scale alongside models whose abilities grow seemingly without limit. We will dissect this issue, along with the core challenges of the above two transitions, from the perspectives of methods, datasets, evaluators, and metrics. Due to the fast evolving of this field, we will maintain a living GitHub repository (links are in each section) to crowd-source updates and corrections, and warmly invite contributors and collaborators.