# The Misuse of AUC: What High Impact Risk Assessment Gets Wrong

arXiv: https://arxiv.org/abs/2305.18159 · HTML: https://mlsanity.com/paper/2305.18159/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 May 2023.

## Built on by (most cited first, 10 of 10)

- [Unlocking Accuracy and Fairness in Differentially Private Image Classification](https://mlsanity.com/paper/2308.10888/) (2023)
- [An end-to-end attention-based approach for learning on graphs](https://mlsanity.com/paper/2402.10793/) (2024)
- [The Neutrality Fallacy: When Algorithmic Fairness Interventions are (Not) Positive Action](https://mlsanity.com/paper/2404.12143/) (2024)
- [Threshold-Independent Fair Matching through Score Calibration](https://mlsanity.com/paper/2405.20051/) (2024)
- [A Consequentialist Critique of Binary Classification Evaluation: Theory, Practice, and Tools](https://mlsanity.com/paper/2504.04528/) (2025)
- [Beyond Predictive Algorithms in Child Welfare](https://mlsanity.com/paper/2403.05573/) (2024)
- [On Fixing the Right Problems in Predictive Analytics: AUC Is Not the Problem](https://mlsanity.com/paper/2404.06989/) (2024)
- [Reducing Biases in Record Matching Through Scores Calibration](https://mlsanity.com/paper/2411.01685/) (2024)
- [Falsifying Discriminant Validity of Predictive Algorithms](https://mlsanity.com/paper/2601.17146/) (2026)
- [When AUC Misleads: Polarization-Aware Evaluation of Deepfake Detectors under Domain Shift](https://mlsanity.com/paper/2606.19184/) (2026)

## Abstract (arXiv)

When determining which machine learning model best performs some high impact risk assessment task, practitioners commonly use the Area under the Curve (AUC) to defend and validate their model choices. In this paper, we argue that the current use and understanding of AUC as a model performance metric misunderstands the way the metric was intended to be used. To this end, we characterize the misuse of AUC and illustrate how this misuse negatively manifests in the real world across several risk assessment domains. We locate this disconnect in the way the original interpretation of AUC has shifted over time to the point where issues pertaining to decision thresholds, class balance, statistical uncertainty, and protected groups remain unaddressed by AUC-based model comparisons, and where model choices that should be the purview of policymakers are hidden behind the veil of mathematical rigor. We conclude that current model validation practices involving AUC are not robust, and often invalid.