mlsanityData to 7 Oct 2026

The Problem with Metrics is a Fundamental Problem for AI

arXiv 2002.08512 ↗First posted 20 Feb 2020cs.CY cs.AIcited by 37

Not on the map. Why?

Abstract (arXiv)

Optimizing a given metric is a central aspect of most current AI approaches, yet overemphasizing metrics leads to manipulation, gaming, a myopic focus on short-term goals, and other unexpected negative consequences. This poses a fundamental contradiction for AI development. Through a series of real-world case studies, we look at various aspects of where metrics go wrong in practice and aspects of how our online environment and current business practices are exacerbating these failures. Finally, we propose a framework towards mitigating the harms caused by overemphasis of metrics within AI by: (1) using a slate of metrics to get a fuller and more nuanced picture, (2) combining metrics with qualitative accounts, and (3) involving a range of stakeholders, including those who will be most impacted.

Key facts. Cited by 37 papers in the map (data through 7 Oct 2026). First posted on arXiv 20 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 8; 2023: 6; 2024: 4; 2025: 7; 2026: 11.

37 papers.

Sort:

By area:

  1. 2022Defining and Characterizing Reward Hackingcited by 184
  2. 2023Large language models can accurately predict searcher preferencescited by 129
  3. 2022Evaluation Gaps in Machine Learning Practicecited by 48
  4. 2022Eliciting and Learning with Soft Labels from Every Annotatorcited by 43
  5. 2021What are you optimizing for? Aligning Recommender Systems with Human Valuescited by 28
  6. 2025Reality Check: A New Evaluation Ecosystem Is Necessary to Understand AI's Real World Effectscited by 18
  7. 2022Aligning Offline Metrics and Human Judgments of Value for Code Generation Modelscited by 14
  8. 2024Promises and pitfalls of artificial intelligence for legal applicationscited by 14
  9. 2023Scaling Laws Do Not Scalecited by 13
  10. 2024Benchmark Data Repositories for Better Benchmarkingcited by 10
  11. 2024GPT for Games: An Updated Scoping Review (2020-2024)cited by 10
  12. 2023Predictable Artificial Intelligencecited by 8
25 more
  1. 2025The Inadequacy of Offline LLM Evaluations: A Need to Account for Personalization in Model Behaviorcited by 8
  2. 2023A Review of the Evidence for Existential Risk from AI via Misaligned Power-Seekingcited by 7
  3. 2024"This is not a data problem": Algorithms and Power in Public Higher Education in Canadacited by 6
  4. 2022Identifying the Context Shift between Test Benchmarks and Production Datacited by 5
  5. 2022Quantitative AI Risk Assessments: Opportunities and Challengescited by 4
  6. 2025Beware of "Explanations" of AIcited by 3
  7. 2023Mapping the Challenges of HCI: An Application and Evaluation of ChatGPT for Mining Insights at Scalecited by 2
  8. 2025Beyond Accuracy: EcoL2 Metric for Sustainable Neural PDE Solverscited by 2
  9. 2025The Strong, Weak and Benign Goodhart's law. An independence-free and paradigm-agnostic formalisationcited by 2
  10. 2022The Different Faces of AI Ethics Across the World: A Principle-Implementation Gap Analysiscited by 1
  11. 2022Challenges in Explanation Quality Evaluationcited by 1
  12. 2023Positive AI: Key Challenges in Designing Artificial Intelligence for Wellbeingcited by 1
  13. 2025Benchmarking and Evaluation of AI Models in Biology: Outcomes and Recommendations from the CZI Virtual Cells Workshopcited by 1
  14. 2026MassSpecGym in the Wild: Uncovering and Correcting Evaluation Pitfalls in AI-Driven Molecule Discoverycited by 1
  15. 2026Against Proxy Optimizationcited by 1
  16. 2026Manipulation-Proof Oblivious Audits against Deceptive Model Providerscited by 1
  17. 2025Branching Out: Broadening AI Measurement and Evaluation with Measurement Treescited by 0
  18. 2026White-Box Sensitivity Auditing with Steering Vectorscited by 0
  19. 2026Rules or Weights? Comparing User Understanding of Explainable AI Techniques with the Cognitive XAI-Adaptive Modelcited by 0
  20. 2026Real-World AI Evaluation: How FRAME Generates Systematic Evidence to Resolve the Decision-Maker's Dilemmacited by 0
  21. 2026VERA-MH: Validation of Ethical and Responsible AI in Mental Healthcited by 0
  22. 2026Position: State-of-the-Art Claims Require State-of-the-Art Evidencecited by 0
  23. 2026Welfare, Improvability, and Variance: A Principal-Agent Approach to Optimal Benchmark Item Aggregationcited by 0
  24. 2026Position: AI Leaderboards Are Underserving the Global South: A Case Study from Indiacited by 0
  25. 2026PhysicsBench: A Unified Leaderboard for Generative and Predictive Models in Engineering Design and Simulationcited by 0