A Conceptual Framework for AI Capability Evaluations
Not on the map. Why?
Abstract (arXiv)
As AI systems advance and integrate into society, well-designed and transparent evaluations are becoming essential tools in AI governance, informing decisions by providing evidence about system capabilities and risks. Yet there remains a lack of clarity on how to perform these assessments both comprehensively and reliably. To address this gap, we propose a conceptual framework for analyzing AI capability evaluations, offering a structured, descriptive approach that systematizes the analysis of widely used methods and terminology without imposing new taxonomies or rigid formats. This framework supports transparency, comparability, and interpretability across diverse evaluations. It also enables researchers to identify methodological weaknesses, assists practitioners in designing evaluations, and provides policymakers with an accessible tool to scrutinize, compare, and navigate complex evaluation landscapes.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 76 of its 101 references that are in the map. First posted on arXiv 23 Jun 2025.
Builds on
76 of 101 references.
References · 76
- 2024Chatbot Arena: An Open Platform for Evaluating LLMs by Human Preferencecited by 1,151
- 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 805
- 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 795
- 2023Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formattingcited by 778
- 2024LLM Evaluators Recognize and Favor Their Own Generationscited by 651
- 2024A Systematic Survey of Prompt Engineering in Large Language Models: Techniques and Applicationscited by 428
- 2023Causal Reasoning and Large Language Models: Opening a New Frontier for Causalitycited by 348
- 2021AI and the Everything in the Whole Wide World Benchmarkcited by 340
- 2023NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmarkcited by 339
- 2023LegalBench: A Collaboratively Built Benchmark for Measuring Legal Reasoning in Large Language Modelscited by 328
- 2023MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedbackcited by 293
- 2024Debating with More Persuasive LLMs Leads to More Truthful Answerscited by 255
38 more
- 2023MathPrompter: Mathematical Reasoning using Large Language Modelscited by 236
- 2023Don't Make Your LLM an Evaluation Benchmark Cheatercited by 194
- 2023Benchmarking Foundation Models with Language-Model-as-an-Examinercited by 172
- 2024Lessons from the Trenches on Reproducible Evaluation of Language Modelscited by 172
- 2025MultiChallenge: A Realistic Multi-Turn Conversation Evaluation Benchmark Challenging to Frontier LLMscited by 171
- 2023Model evaluation for extreme riskscited by 167
- 2022Repairing the Cracked Foundation: A Survey of Obstacles in Evaluation Practices for Generated Textcited by 154
- 2023Sociotechnical Safety Evaluation of Generative AI Systemscited by 154
- 2023Can Large Language Models Infer Causation from Correlation?cited by 142
- 2023Stop Uploading Test Data in Plain Text: Practical Strategies for Mitigating Data Contamination by Evaluation Benchmarkscited by 139
- 2023Auditing large language models: a three-layered approachcited by 135
- 2024Adding Error Bars to Evals: A Statistical Approach to Language Model Evaluationscited by 130
- 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 123
- 2024Generalization or Memorization: Data Contamination and Trustworthy Evaluation for Large Language Modelscited by 119
- 2024BetterBench: Assessing AI Benchmarks, Uncovering Issues, and Establishing Best Practicescited by 112
- 2023CLadder: Assessing Causal Reasoning in Language Modelscited by 102
- 2023Deception Abilities Emerged in Large Language Modelscited by 99
- 2024A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendationscited by 84
- 2024Curiosity-driven Red-teaming for Large Language Modelscited by 83
- 2024Metacognitive Capabilities of LLMs: An Exploration in Mathematical Problem Solvingcited by 82
- 2024LLM-based NLG Evaluation: Current Status and Challengescited by 71
- 2024Datasets for Large Language Models: A Comprehensive Surveycited by 69
- 2024Inadequacies of Large Language Model Benchmarks in the Era of Generative Artificial Intelligencecited by 65
- 2019Ablation Studies in Artificial Neural Networkscited by 58
- 2025Can We Trust AI Benchmarks? An Interdisciplinary Review of Current Issues in AI Evaluationcited by 57
- 2023Rethinking Model Evaluation as Narrowing the Socio-Technical Gapcited by 56
- 2021Comparing Test Sets with Item Response Theorycited by 55
- 2024Against The Achilles' Heel: A Survey on Red Teaming for Generative Modelscited by 55
- 2024Learning diverse attacks on large language models for robust red-teaming and safety tuningcited by 54
- 2022On the Limitations of Reference-Free Evaluations of Generated Textcited by 52
- 2020Targeting the Benchmark: On Methodology in Current Natural Language Processing Researchcited by 44
- 2025Toward an Evaluation Science for Generative AI Systemscited by 42
- 2024Leveraging Large Language Models for NLG Evaluation: Advances and Challengescited by 37
- 2025LessLeak-Bench: A First Investigation of Data Leakage in LLMs Across 83 Software Engineering Benchmarkscited by 34
- 2022Deconstructing NLG Evaluation: Evaluation Practices, Assumptions, and Their Implicationscited by 32
- 2024Leveraging the Context through Multi-Round Interactions for Jailbreaking Attackscited by 32
- 2023The ROOTS Search Tool: Data Transparency for LLMscited by 30
- 2023LLMEval: A Preliminary Study on How to Evaluate Large Language Modelscited by 23
50 most cited of 76 All 76 (JSON)
Built on by
Not cited yet.