# A Closer Look at AUROC and AUPRC under Class Imbalance

arXiv: https://arxiv.org/abs/2401.06091 · HTML: https://mlsanity.com/paper/2401.06091/

## Key facts

- Cited by 57 papers in the map (data through 14 Oct 2026).
- Builds on 101 of its 169 references that are in the map.
- First posted on arXiv 11 Jan 2024.

## Builds on (most cited first, 30 of 101)

- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Mitigating Unwanted Biases with Adversarial Learning](https://mlsanity.com/paper/1801.07593/) (2018)
- [DRAEM -- A discriminatively trained reconstruction embedding for surface anomaly detection](https://mlsanity.com/paper/2108.07610/) (2021)
- [A Unifying Review of Deep and Shallow Anomaly Detection](https://mlsanity.com/paper/2009.11732/) (2020)
- [SPot-the-Difference Self-Supervised Pre-training for Anomaly Detection and Segmentation](https://mlsanity.com/paper/2207.14315/) (2022)
- [Multitask learning and benchmarking with clinical time series data](https://mlsanity.com/paper/1703.07771/) (2017)
- [Fairness without Demographics through Adversarially Reweighted Learning](https://mlsanity.com/paper/2006.13114/) (2020)
- [Not only Look, but also Listen: Learning Multimodal Violence Detection under Weak Supervision](https://mlsanity.com/paper/2007.04687/) (2020)
- [Foundations and modelling of dynamic networks using Dynamic Graph Neural Networks: A survey](https://mlsanity.com/paper/2005.07496/) (2020)
- [Interpolation-Prediction Networks for Irregularly Sampled Time Series](https://mlsanity.com/paper/1909.07782/) (2019)
- [MiME: Multilevel Medical Embedding of Electronic Health Records for Predictive Healthcare](https://mlsanity.com/paper/1810.09593/) (2018)
- [Predicting Deep Zero-Shot Convolutional Neural Networks using Textual Descriptions](https://mlsanity.com/paper/1506.00511/) (2015)
- [Deep Over-sampling Framework for Classifying Imbalanced Data](https://mlsanity.com/paper/1704.07515/) (2017)
- [Deep Weakly-supervised Anomaly Detection](https://mlsanity.com/paper/1910.13601/) (2019)
- [ConCare: Personalized Clinical Feature Embedding via Capturing the Healthcare Context](https://mlsanity.com/paper/1911.12216/) (2019)
- [Differentially Private Synthetic Medical Data Generation using Convolutional GANs](https://mlsanity.com/paper/2012.11774/) (2020)
- [Detecting semantic anomalies](https://mlsanity.com/paper/1908.04388/) (2019)
- [An overview of deep learning based methods for unsupervised and semi-supervised anomaly detection in videos](https://mlsanity.com/paper/1801.03149/) (2018)
- [Automated software vulnerability detection with machine learning](https://mlsanity.com/paper/1803.04497/) (2018)
- [Evaluating Link Prediction Methods](https://mlsanity.com/paper/1505.04094/) (2015)
- [Toward Interpretable Music Tagging with Self-Attention](https://mlsanity.com/paper/1906.04972/) (2019)
- [Molecular machine learning with conformer ensembles](https://mlsanity.com/paper/2012.08452/) (2020)
- [Benchmarking Unsupervised Outlier Detection with Realistic Synthetic Data](https://mlsanity.com/paper/2004.06947/) (2020)
- [Multimodal Machine Learning-based Knee Osteoarthritis Progression Prediction from Plain Radiographs and Clinical Data](https://mlsanity.com/paper/1904.06236/) (2019)
- [Asymmetric Loss Functions and Deep Densely Connected Networks for Highly Imbalanced Medical Image Segmentation: Application to Multiple Sclerosis Lesion Detection](https://mlsanity.com/paper/1803.11078/) (2018)
- [RAPID: Early Classification of Explosive Transients using Deep Learning](https://mlsanity.com/paper/1904.00014/) (2019)
- [Net benefit, calibration, threshold selection, and training objectives for algorithmic fairness in healthcare](https://mlsanity.com/paper/2202.01906/) (2022)
- [Characterizing the Value of Information in Medical Notes](https://mlsanity.com/paper/2010.03574/) (2020)
- [Fail-Safe Execution of Deep Learning based Systems through Uncertainty Monitoring](https://mlsanity.com/paper/2102.00902/) (2021)
- [A Revealing Large-Scale Evaluation of Unsupervised Anomaly Detection Algorithms](https://mlsanity.com/paper/2204.09825/) (2022)

## Built on by (most cited first, 30 of 57)

- [Developing ChemDFM as a large language foundation model for chemistry](https://mlsanity.com/paper/2401.14818/) (2024)
- [REALM: RAG-Driven Enhancement of Multimodal Electronic Health Records Analysis via Large Language Models](https://mlsanity.com/paper/2402.07016/) (2024)
- [ColaCare: Enhancing Electronic Health Record Modeling through Large Language Model-Driven Multi-Agent Collaboration](https://mlsanity.com/paper/2410.02551/) (2024)
- [EMERGE: Enhancing Multimodal Electronic Health Records Predictive Modeling with Retrieval-Augmented Generation](https://mlsanity.com/paper/2406.00036/) (2024)
- [Enhancing clinical decision support with physiological waveforms -- a multimodal benchmark in emergency care](https://mlsanity.com/paper/2407.17856/) (2024)
- [Training a high-performance retinal foundation model with half-the-data and 400 times less compute](https://mlsanity.com/paper/2405.00117/) (2024)
- [TimEHR: Image-based Time Series Generation for Electronic Health Records](https://mlsanity.com/paper/2402.06318/) (2024)
- [A Collection of Innovations in Medical AI for patient records in 2024](https://mlsanity.com/paper/2503.05768/) (2025)
- [Benchmarking ECG FMs: A Reality Check Across Clinical Tasks](https://mlsanity.com/paper/2509.25095/) (2025)
- [Learning Contrastive Multimodal Fusion with Improved Modality Dropout for Disease Detection and Prediction](https://mlsanity.com/paper/2509.18284/) (2025)
- [Not All Inputs Are Valid: Towards Open-Set Video Moment Retrieval Using Language](https://mlsanity.com/paper/2605.29812/) (2026)
- [Abnormality Prediction and Forecasting of Laboratory Values from Electrocardiogram Signals Using Multimodal Deep Learning](https://mlsanity.com/paper/2411.14886/) (2024)
- [Cohort-Based Active Modality Acquisition](https://mlsanity.com/paper/2505.16791/) (2025)
- [Explaining Time Series Classification Predictions via Causal Attributions](https://mlsanity.com/paper/2405.15871/) (2024)
- [We Need to Rethink Benchmarking in Anomaly Detection](https://mlsanity.com/paper/2507.15584/) (2025)
- [Synthetic Data Generation for Augmenting Small Samples](https://mlsanity.com/paper/2501.18741/) (2025)
- [Likert or Not: LLM Absolute Relevance Judgments on Fine-Grained Ordinal Scales](https://mlsanity.com/paper/2505.19334/) (2025)
- [Aligning Evaluation with Clinical Priorities: Calibration, Label Shift, and Error Costs](https://mlsanity.com/paper/2506.14540/) (2025)
- [DACTYL: Diverse Adversarial Corpus of Texts Yielded from Large Language Models](https://mlsanity.com/paper/2508.00619/) (2025)
- [ReTabAD: A Benchmark for Restoring Semantic Context in Tabular Anomaly Detection](https://mlsanity.com/paper/2510.02060/) (2025)
- [Rethinking Tokenization for Clinical Time Series: When Less is More](https://mlsanity.com/paper/2512.05217/) (2025)
- [When Deep Research Agents Stagnate: Enhancing Reasoning with Retrieval-Aware Agent Control](https://mlsanity.com/paper/2608.15191/) (2026)
- [Critical Appraisal of Fairness Metrics in Clinical Predictive AI](https://mlsanity.com/paper/2506.17035/) (2025)
- [Limitations of Public Chest Radiography Datasets for Artificial Intelligence: Label Quality, Domain Shift, Bias and Evaluation Challenges](https://mlsanity.com/paper/2509.15107/) (2025)
- [Traj-CoA: Patient Trajectory Modeling via Chain-of-Agents for Lung Cancer Risk Prediction](https://mlsanity.com/paper/2510.10454/) (2025)
- [A multi-agent system for spine MRI report generation from multi-sequence imaging](https://mlsanity.com/paper/2606.08897/) (2026)
- [Med-gte-hybrid: A contextual embedding transformer model for extracting actionable information from clinical texts](https://mlsanity.com/paper/2502.15996/) (2025)
- [The Role of Model Confidence on Bias Effects in Measured Uncertainties for Vision-Language Models](https://mlsanity.com/paper/2506.16724/) (2025)
- [Unsupervised Deep Generative Models for Anomaly Detection in Neuroimaging: A Systematic Scoping Review](https://mlsanity.com/paper/2510.14462/) (2025)
- [KAT-GNN: A Knowledge-Augmented Temporal Graph Neural Network for Risk Prediction in Electronic Health Records](https://mlsanity.com/paper/2511.01249/) (2025)

## Abstract (arXiv)

In machine learning (ML), a widespread claim is that the area under the precision-recall curve (AUPRC) is a superior metric for model comparison to the area under the receiver operating characteristic (AUROC) for tasks with class imbalance. This paper refutes this notion on two fronts. First, we theoretically characterize the behavior of AUROC and AUPRC in the presence of model mistakes, establishing clearly that AUPRC is not generally superior in cases of class imbalance. We further show that AUPRC can be a harmful metric as it can unduly favor model improvements in subpopulations with more frequent positive labels, heightening algorithmic disparities. Next, we empirically support our theory using experiments on both semi-synthetic and real-world fairness datasets. Prompted by these insights, we conduct a review of over 1.5 million scientific papers to understand the origin of this invalid claim, finding that it is often made without citation, misattributed to papers that do not argue this point, and aggressively over-generalized from source arguments. Our findings represent a dual contribution: a significant technical advancement in understanding the relationship between AUROC and AUPRC and a stark warning about unchecked assumptions in the ML community.