mlsanityData to 14 Oct 2026

Paradigms of AI Evaluation: Mapping Goals, Methodologies and Culture

arXiv 2502.15620 ↗First posted 21 Feb 2025cs.AI cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

Research in AI evaluation has grown increasingly complex and multidisciplinary, attracting researchers with diverse backgrounds and objectives. As a result, divergent evaluation paradigms have emerged, often developing in isolation, adopting conflicting terminologies, and overlooking each other's contributions. This fragmentation has led to insular research trajectories and communication barriers both among different paradigms and with the general public, contributing to unmet expectations for deployed AI systems. To help bridge this insularity, in this paper we survey recent work in the AI evaluation landscape and identify six main paradigms. We characterise major recent contributions within each paradigm across key dimensions related to their goals, methodologies and research cultures. By clarifying the unique combination of questions and approaches associated with each paradigm, we aim to increase awareness of the breadth of current evaluation approaches and foster cross-pollination between different paradigms. We also identify potential gaps in the field to inspire future research directions.

Key facts. Cited by 11 papers in the map (data through 14 Oct 2026). Builds on 32 of its 55 references that are in the map. First posted on arXiv 21 Feb 2025.

Builds on

32 of 55 references.

References · 32

  1. 2020Measuring Massive Multitask Language Understandingcited by 7,963
  2. 2020Scaling Laws for Neural Language Modelscited by 6,586
  3. 2017Deep reinforcement learning from human preferencescited by 3,912
  4. 2018UMAP: Uniform Manifold Approximation and Projection for Dimension Reductioncited by 2,898
  5. 2023A Survey on Evaluation of Large Language Modelscited by 1,242
  6. 2022Discovering Language Model Behaviors with Model-Written Evaluationscited by 836
  7. 2022Red Teaming Language Models to Reduce Harms: Methods, Scaling Behaviors, and Lessons Learnedcited by 729
  8. 2024Mechanistic Interpretability for AI Safety -- A Reviewcited by 422
  9. 2023The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A"cited by 407
  10. 2024Can LLMs Generate Novel Research Ideas? A Large-Scale Human Study with 100+ NLP Researcherscited by 363
  11. 2021AI and the Everything in the Whole Wide World Benchmarkcited by 349
  12. 2024tinyBenchmarks: evaluating LLMs with fewer examplescited by 310
20 more
  1. 2021Unsolved Problems in ML Safetycited by 292
  2. 2023Model evaluation for extreme riskscited by 177
  3. 2024Evaluating Frontier Models for Dangerous Capabilitiescited by 128
  4. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 126
  5. 2024What Large Language Models Know and What People Think They Knowcited by 110
  6. 2023Evaluating Language-Model Agents on Realistic Autonomous Taskscited by 96
  7. 2023A Survey of Safety and Trustworthiness of Large Language Models through the Lens of Verification and Validationcited by 93
  8. 2023Evaluating Cognitive Maps and Planning in Large Language Models with CogEvalcited by 77
  9. 2023Evaluating Language Models for Mathematics through Interactionscited by 70
  10. 2024CogBench: a large language model walks into a psychology labcited by 63
  11. 2024Against The Achilles' Heel: A Survey on Red Teaming for Generative Modelscited by 59
  12. 2022Evaluation Gaps in Machine Learning Practicecited by 49
  13. 2023Safe Reinforcement Learning via Probabilistic Logic Shieldscited by 42
  14. 2022Melting Pot 2.0cited by 40
  15. 2023Does progress on ImageNet transfer to real-world datasets?cited by 35
  16. 2023It Takes Two to Tango: Navigating Conceptualizations of NLP Tasks and Measurements of Performancecited by 26
  17. 2024Automated Evaluation of Retrieval-Augmented Language Models with Task-Specific Exam Generationcited by 25
  18. 2024Can a Bayesian Oracle Prevent Harm from an Agent?cited by 13
  19. 2024From Protoscience to Epistemic Monoculture: How Benchmarking Set the Stage for the Deep Learning Revolutioncited by 12
  20. 2024Evaluating AI Evaluation: Perils and Prospectscited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 5.

11 papers.

  1. 2026Capabilities Ain't All You Need: Measuring Propensities in AIcited by 6
  2. 2025Measuring Data Science Automation: A Survey of Evaluation Tools for AI Assistants and Agentscited by 4
  3. 2025Statistical Methods in Generative AIcited by 4
  4. 2025The Artificial Intelligence Cognitive Examination: A Survey on the Evolution of Multimodal Evaluation from Recognition to Reasoningcited by 4
  5. 2026From Human-Level AI Tales to AI Leveling Human Scalescited by 1
  6. 2025A Conceptual Framework for AI Capability Evaluationscited by 0
  7. 2025Towards Integrated Alignmentcited by 0
  8. 2025Fluidity Index: Next-Generation Super-intelligence Benchmarkscited by 0
  9. 2026A Multi-Turn Framework for Evaluating AI Misuse in Fraud and Cybercrime Scenarioscited by 0
  10. 2026RCTs for Frontier AI Governance: Methodological Challenges and Solutions for Human Uplift Studiescited by 0
  11. 2026A 3D Characterization Framework for Intelligent Sequential Decision Makingcited by 0