# Deep ROC Analysis and AUC as Balanced Average Accuracy to Improve Model Selection, Understanding and Interpretation

arXiv: https://arxiv.org/abs/2103.11357 · HTML: https://mlsanity.com/paper/2103.11357/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 100 references that are in the map.
- First posted on arXiv 21 Mar 2021.

## Builds on (most cited first, 7 of 7)

- [A Survey on Explainable Artificial Intelligence (XAI): Towards Medical XAI](https://mlsanity.com/paper/1907.07374/) (2019)
- [Closing the AI Accountability Gap: Defining an End-to-End Framework for Internal Algorithmic Auditing](https://mlsanity.com/paper/2001.00973/) (2020)
- [Salient Object Detection in the Deep Learning Era: An In-Depth Survey](https://mlsanity.com/paper/1904.09146/) (2019)
- [Measuring the Quality of Explanations: The System Causability Scale (SCS). Comparing Human and Machine Explanations](https://mlsanity.com/paper/1912.09024/) (2019)
- [Pairwise Fairness for Ranking and Regression](https://mlsanity.com/paper/1906.05330/) (2019)
- [The Fairness of Risk Scores Beyond Classification: Bipartite Ranking and the xAUC Metric](https://mlsanity.com/paper/1902.05826/) (2019)
- [Two-Way Partial AUC and Its Properties](https://mlsanity.com/paper/1508.00298/) (2015)

## Built on by (most cited first, 17 of 17)

- [Word-Sequence Entropy: Towards Uncertainty Estimation in Free-Form Medical Question Answering Applications and Beyond](https://mlsanity.com/paper/2402.14259/) (2024)
- [Aligning Evaluation with Clinical Priorities: Calibration, Label Shift, and Error Costs](https://mlsanity.com/paper/2506.14540/) (2025)
- [Membership Inference Attacks on Tokenizers of Large Language Models](https://mlsanity.com/paper/2510.05699/) (2025)
- [Design Space Exploration of Hybrid Quantum Neural Networks for Chronic Kidney Disease](https://mlsanity.com/paper/2604.13608/) (2026)
- [A Consequentialist Critique of Binary Classification Evaluation: Theory, Practice, and Tools](https://mlsanity.com/paper/2504.04528/) (2025)
- [Decomposing Global AUC into Cluster-Level Contributions for Localized Model Diagnostics](https://mlsanity.com/paper/2508.07495/) (2025)
- [Towards actionable hypotension prediction -- predicting catecholamine therapy initiation in the intensive care unit](https://mlsanity.com/paper/2510.24287/) (2025)
- [Anomaly Detection in Intra-Vehicle Networks](https://mlsanity.com/paper/2205.03537/) (2022)
- [Transforming Triple-Entry Accounting with Machine Learning: A Path to Enhanced Transparency Through Analytics](https://mlsanity.com/paper/2411.15190/) (2024)
- [DCS-ST for Classification of Breast Cancer Histopathology Images with Limited Annotations](https://mlsanity.com/paper/2505.03204/) (2025)
- [SubROC: AUC-Based Discovery of Exceptional Subgroup Performance for Binary Classifiers](https://mlsanity.com/paper/2505.11283/) (2025)
- [CT Radiomics-Based Explainable Machine Learning Model for Accurate Differentiation of Malignant and Benign Endometrial Tumors: A Two-Center Study](https://mlsanity.com/paper/2506.18106/) (2025)
- [Automated Landfill Detection Using Deep Learning: A Comparative Study of Lightweight and Custom Architectures with the AerialWaste Dataset](https://mlsanity.com/paper/2508.18315/) (2025)
- [Predicting Time Pressure of Powered Two-Wheeler Riders for Proactive Safety Interventions](https://mlsanity.com/paper/2601.03173/) (2026)
- [GAFR-Net: A Graph Attention and Fuzzy-Rule Network for Interpretable Breast Cancer Image Classification](https://mlsanity.com/paper/2602.09318/) (2026)
- [MRC-GAT: A Meta-Relational Copula-Based Graph Attention Network for Interpretable Multimodal Alzheimer's Disease Diagnosis](https://mlsanity.com/paper/2602.15740/) (2026)
- [ChemCLIP: Bridging Organic and Inorganic Anticancer Compounds Through Contrastive Learning](https://mlsanity.com/paper/2603.28575/) (2026)

## Abstract (arXiv)

Optimal performance is critical for decision-making tasks from medicine to autonomous driving, however common performance measures may be too general or too specific. For binary classifiers, diagnostic tests or prognosis at a timepoint, measures such as the area under the receiver operating characteristic curve, or the area under the precision recall curve, are too general because they include unrealistic decision thresholds. On the other hand, measures such as accuracy, sensitivity or the F1 score are measures at a single threshold that reflect an individual single probability or predicted risk, rather than a range of individuals or risk. We propose a method in between, deep ROC analysis, that examines groups of probabilities or predicted risks for more insightful analysis. We translate esoteric measures into familiar terms: AUC and the normalized concordant partial AUC are balanced average accuracy (a new finding); the normalized partial AUC is average sensitivity; and the normalized horizontal partial AUC is average specificity. Along with post-test measures, we provide a method that can improve model selection in some cases and provide interpretation and assurance for patients in each risk group. We demonstrate deep ROC analysis in two case studies and provide a toolkit in Python.