# On Goodhart's law, with an application to value alignment

arXiv: https://arxiv.org/abs/2410.09638 · HTML: https://mlsanity.com/paper/2410.09638/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 50 references that are in the map.
- First posted on arXiv 12 Oct 2024.

## Builds on (most cited first, 21 of 21)

- [Understanding deep learning requires rethinking generalization](https://mlsanity.com/paper/1611.03530/) (2016)
- [Concrete Problems in AI Safety](https://mlsanity.com/paper/1606.06565/) (2016)
- [Reconciling modern machine learning practice and the bias-variance trade-off](https://mlsanity.com/paper/1812.11118/) (2018)
- [Deep Double Descent: Where Bigger Models and More Data Hurt](https://mlsanity.com/paper/1912.02292/) (2019)
- [Surprises in High-Dimensional Ridgeless Least Squares Interpolation](https://mlsanity.com/paper/1903.08560/) (2019)
- [The generalization error of random features regression: Precise asymptotics and double descent curve](https://mlsanity.com/paper/1908.05355/) (2019)
- [Robust Estimators in High Dimensions without the Computational Intractability](https://mlsanity.com/paper/1604.06443/) (2016)
- [Two models of double descent for weak features](https://mlsanity.com/paper/1903.07571/) (2019)
- [To understand deep learning we need to understand kernel learning](https://mlsanity.com/paper/1802.01396/) (2018)
- [Categorizing Variants of Goodhart's Law](https://mlsanity.com/paper/1803.04585/) (2018)
- [Harmless interpolation of noisy data in regression](https://mlsanity.com/paper/1903.09139/) (2019)
- [Optimal Policies Tend to Seek Power](https://mlsanity.com/paper/1912.01683/) (2019)
- [Does data interpolation contradict statistical optimality?](https://mlsanity.com/paper/1806.09471/) (2018)
- [Incomplete Contracting and AI Alignment](https://mlsanity.com/paper/1804.04268/) (2018)
- [A Voting-Based System for Ethical Decision Making](https://mlsanity.com/paper/1709.06692/) (2017)
- [Auditing Radicalization Pathways on YouTube](https://mlsanity.com/paper/1908.08313/) (2019)
- [Loss minimization and parameter estimation with heavy tails](https://mlsanity.com/paper/1307.1827/) (2013)
- [Robust subgaussian estimation of a mean vector in nearly linear time](https://mlsanity.com/paper/1906.03058/) (2019)
- [Adapting a Kidney Exchange Algorithm to Align with Human Values](https://mlsanity.com/paper/2005.09755/) (2020)
- [Why are Adaptive Methods Good for Attention Models?](https://mlsanity.com/paper/1912.03194/) (2019)
- [Embedded Agency](https://mlsanity.com/paper/1902.09469/) (2019)

## Built on by (most cited first, 9 of 9)

- [Inference-Time Reward Hacking in Large Language Models](https://mlsanity.com/paper/2506.19248/) (2025)
- [The Reward Model Selection Crisis in Personalized Alignment](https://mlsanity.com/paper/2512.23067/) (2025)
- [The Strong, Weak and Benign Goodhart's law. An independence-free and paradigm-agnostic formalisation](https://mlsanity.com/paper/2505.23445/) (2025)
- [Consequentialist Objectives and Catastrophe](https://mlsanity.com/paper/2603.15017/) (2026)
- [Intrinsic Selection and Particle Resampling for Inference-Time Scaling Beyond Domain Verifiability](https://mlsanity.com/paper/2606.08850/) (2026)
- [A Case for Specialisation in Non-Human Entities](https://mlsanity.com/paper/2503.04742/) (2025)
- [Take Goodhart Seriously: Principled Limit on General-Purpose AI Optimization](https://mlsanity.com/paper/2510.02840/) (2025)
- [Why Code, Why Now: An Information-Theoretic Perspective on the Limits of Machine Learning](https://mlsanity.com/paper/2602.13934/) (2026)
- [A Cross Community Agenda for Speech AI](https://mlsanity.com/paper/2609.13168/) (2026)

## Abstract (arXiv)

``When a measure becomes a target, it ceases to be a good measure'', this adage is known as it Goodhart's law. In this paper, we investigate formally this law and prove that it critically depends on the tail distribution of the discrepancy between the true goal and the measure that is optimized. Discrepancies with long-tail distributions favor a Goodhart's law, that is, the optimization of the measure can have a counter-productive effect on the goal. We provide a formal setting to assess Goodhart's law by studying the asymptotic behavior of the correlation between the goal and the measure, as the measure is optimized. Moreover, we introduce a distinction between a it weak Goodhart's law, when over-optimizing the metric is useless for the true goal, and a it strong Goodhart's law, when over-optimizing the metric is harmful for the true goal. A distinction which we prove to depend on the tail distribution. We stress the implications of this result to large-scale decision making and policies that are (and have to be) based on metrics, and propose numerous research directions to better assess the safety of such policies in general, and to the particularly concerning case where these policies are automated with algorithms.