mlsanityData to 7 Oct 2026

Categorizing Variants of Goodhart's Law

arXiv 1803.04585 ↗First posted 13 Mar 2018cs.AI q-fin.GN stat.MLcited by 132

Not on the map. Why?

Cited by 32 in LLM alignment / RLHF / preference optimisation, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

There are several distinct failure modes for overoptimization of systems on the basis of metrics. This occurs when a metric which can be used to improve a system is used to an extent that further optimization is ineffective or harmful, and is sometimes termed Goodhart's Law. This class of failure is often poorly understood, partly because terminology for discussing them is ambiguous, and partly because discussion using this ambiguous terminology ignores distinctions between different failure modes of this general type. This paper expands on an earlier discussion by Garrabrant, which notes there are "(at least) four different mechanisms" that relate to Goodhart's Law. This paper is intended to explore these mechanisms further, and specify more clearly how they occur. This discussion should be helpful in better understanding these types of failures in economic regulation, in public policy, in machine learning, and in Artificial Intelligence alignment. The importance of Goodhart effects depends on the amount of power directed towards optimizing the proxy, and so the increased optimization power offered by artificial intelligence makes it especially critical for that field.

Key facts. Cited by 132 papers in the map (data through 7 Oct 2026). First posted on arXiv 13 Mar 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2018: 1; 2019: 0; 2020: 0; 2021: 3; 2022: 11; 2023: 9; 2024: 14; 2025: 14; 2026: 80.

132 papers.

Sort:

By area:

  1. 2022Scaling Laws for Reward Model Overoptimizationcited by 1,056
  2. 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 743
  3. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 711
  4. 2022Self-critiquing models for assisting human evaluatorscited by 372
  5. 2021Unsolved Problems in ML Safetycited by 291
  6. 2022The Alignment Problem from a Deep Learning Perspectivecited by 246
  7. 2024Disentangling Length from Quality in Direct Preference Optimizationcited by 194
  8. 2022Defining and Characterizing Reward Hackingcited by 184
  9. 2024Scaling Laws for Reward Model Overoptimization in Direct Alignment Algorithmscited by 139
  10. 2021Truthful AI: Developing and governing AI that does not liecited by 109
  11. 2022Is Power-Seeking AI an Existential Risk?cited by 101
  12. 2024Towards Guaranteed Safe AI: A Framework for Ensuring Robust and Reliable AI Systemscited by 94
38 more
  1. 2021Consequences of Misaligned AIcited by 79
  2. 2024Correlated Proxies: A New Definition and Improved Mitigation for Reward Hackingcited by 50
  3. 2023Goodhart's Law in Reinforcement Learningcited by 37
  4. 2024Style Outweighs Substance: Failure Modes of LLM Judges in Alignment Benchmarkingcited by 35
  5. 2023The Trickle-down Impact of Reward (In-)consistency on RLHFcited by 29
  6. 2024Rethinking Reward Model Evaluation: Are We Barking up the Wrong Tree?cited by 29
  7. 2026SpecBench: Measuring Reward Hacking in Long-Horizon Coding Agentscited by 25
  8. 2022Law Informs Code: A Legal Informatics Approach to Aligning Artificial Intelligence with Humanscited by 23
  9. 2022The four-fifths rule is not disparate impact: a woeful tale of epistemic trespassing in algorithmic fairnesscited by 21
  10. 2025Adaptation of Agentic AI: A Survey of Post-Training, Memory, and Skillscited by 19
  11. 2024A Philosophical Introduction to Language Models - Part II: The Way Forwardcited by 17
  12. 2025Rethinking Reward Model Evaluation Through the Lens of Reward Overoptimizationcited by 10
  13. 2022A taxonomic system for failure cause analysis of open source AI incidentscited by 9
  14. 2024On Goodhart's law, with an application to value alignmentcited by 9
  15. 2026When LLM Judge Scores Look Good but Best-of-N Decisions Failcited by 8
  16. 2023A Review of the Evidence for Existential Risk from AI via Misaligned Power-Seekingcited by 7
  17. 2024Collaboration Dynamics and Reliability Challenges of Multi-Agent LLM Systems in Finite Element Analysiscited by 6
  18. 2025AI Testing Should Account for Sophisticated Strategic Behaviourcited by 6
  19. 2026The Verification Horizon: No Silver Bullet for Coding Agent Rewardscited by 6
  20. 2026Do Agent Optimizers Compound? A Continual-Learning Evaluation on Terminal-Bench 2.0cited by 6
  21. 2026Do Agent Benchmarks Measure Capability? Protocol Validity in the Age of Agentic AIcited by 6
  22. 2023An International Consortium for Evaluations of Societal-Scale Risks from Advanced AIcited by 5
  23. 2024Philosophy of Cognitive Science in the Age of Deep Learningcited by 5
  24. 2026SARC: A Governance-by-Architecture Framework for Agentic AI Systemscited by 5
  25. 2022PEPPER: Empowering User-Centric Recommender Systems over Gossip Learningcited by 4
  26. 2023Intent-aligned AI systems deplete human agency: the need for agency foundations research in AI safetycited by 4
  27. 2024Elephant in the Room: Unveiling the Impact of Reward Model Quality in Alignmentcited by 4
  28. 2024Partial Identifiability and Misspecification in Inverse Reinforcement Learningcited by 4
  29. 2025Scalable Oversight for Superhuman AI via Recursive Self-Critiquingcited by 4
  30. 2025Murphys Laws of AI Alignment: Why the Gap Always Winscited by 4
  31. 2026Preventing the Collapse of Peer Review Requires Verification-First AIcited by 3
  32. 2026Interactive Evaluation Requires a Design Sciencecited by 3
  33. 2026Large Language Models Hack Rewards, and Societycited by 3
  34. 2026Building to the Test: Coding Agents Deliver What You Check, Not What You Requestedcited by 3
  35. 2023Science in the Era of ChatGPT, Large Language Models and Generative AI: Challenges for Research Ethics and How to Respondcited by 2
  36. 2025The Economics of p(doom): Scenarios of Existential Risk and Economic Growth in the Age of Transformative AIcited by 2
  37. 2025One Goal, Many Challenges: Robust Preference Optimization Amid Content-Aware and Multi-Source Noisecited by 2
  38. 2025The Strong, Weak and Benign Goodhart's law. An independence-free and paradigm-agnostic formalisationcited by 2

50 most cited of 132 All 132 (JSON)