# Overcoming Common Flaws in the Evaluation of Selective Classification Systems

arXiv: https://arxiv.org/abs/2407.01032 · HTML: https://mlsanity.com/paper/2407.01032/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 57 references that are in the map.
- First posted on arXiv 1 Jul 2024.

## Builds on (most cited first, 30 of 39)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [Selective Classification for Deep Neural Networks](https://mlsanity.com/paper/1705.08500/) (2017)
- [Scaling Vision Transformers to 22 Billion Parameters](https://mlsanity.com/paper/2302.05442/) (2023)
- [SelectiveNet: A Deep Neural Network with an Integrated Reject Option](https://mlsanity.com/paper/1901.09192/) (2019)
- [Learning Confidence for Out-of-Distribution Detection in Neural Networks](https://mlsanity.com/paper/1802.04865/) (2018)
- [Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learning](https://mlsanity.com/paper/2002.06470/) (2020)
- [Addressing Failure Prediction by Learning Model Confidence](https://mlsanity.com/paper/1910.04851/) (2019)
- [BREEDS: Benchmarks for Subpopulation Shift](https://mlsanity.com/paper/2008.04859/) (2020)
- [Document Understanding Dataset and Evaluation (DUDE)](https://mlsanity.com/paper/2305.08455/) (2023)
- [Bias-Reduced Uncertainty Estimation for Deep Neural Classifiers](https://mlsanity.com/paper/1805.08206/) (2018)
- [Plex: Towards Reliability using Pretrained Large Model Extensions](https://mlsanity.com/paper/2207.07411/) (2022)
- [Shifts: A Dataset of Real Distributional Shift Across Multiple Large-Scale Tasks](https://mlsanity.com/paper/2107.07455/) (2021)
- [Deep Gamblers: Learning to Abstain with Portfolio Theory](https://mlsanity.com/paper/1907.00208/) (2019)
- [Optimal strategies for reject option classifiers](https://mlsanity.com/paper/2101.12523/) (2021)
- [Methods and open-source toolkit for analyzing and visualizing challenge results](https://mlsanity.com/paper/1910.05121/) (2019)
- [A Call to Reflect on Evaluation Practices for Failure Detection in Image Classification](https://mlsanity.com/paper/2211.15259/) (2022)
- [Rethinking Confidence Calibration for Failure Prediction](https://mlsanity.com/paper/2303.02970/) (2023)
- [OpenMix: Exploring Outlier Samples for Misclassification Detection](https://mlsanity.com/paper/2303.17093/) (2023)
- [What Can We Learn From The Selective Prediction And Uncertainty Estimation Performance Of 523 Imagenet Classifiers](https://mlsanity.com/paper/2302.11874/) (2023)
- [Augmenting Softmax Information for Selective Classification with Out-of-Distribution Data](https://mlsanity.com/paper/2207.07506/) (2022)
- [Towards Better Selective Classification](https://mlsanity.com/paper/2206.09034/) (2022)
- [Towards Improving Selective Prediction Ability of NLP Systems](https://mlsanity.com/paper/2008.09371/) (2020)
- [Disrupting Deep Uncertainty Estimation Without Harming Accuracy](https://mlsanity.com/paper/2110.13741/) (2021)
- [A Unified Benchmark for the Unknown Detection Capability of Deep Neural Networks](https://mlsanity.com/paper/2112.00337/) (2021)
- [Measuring and Improving Model-Moderator Collaboration using Uncertainty Estimation](https://mlsanity.com/paper/2107.04212/) (2021)

## Built on by (most cited first, 30 of 37)

- [A Novel Characterization of the Population Area Under the Risk Coverage Curve (AURC) and Rates of Finite Sample Estimators](https://mlsanity.com/paper/2410.15361/) (2024)
- [The Cost of Reasoning: Chain-of-Thought Induces Overconfidence in Vision-Language Models](https://mlsanity.com/paper/2603.16728/) (2026)
- [Why context matters in VQA and Reasoning: Semantic interventions for VLM input modalities](https://mlsanity.com/paper/2410.01690/) (2024)
- [What Does It Take to Build a Performant Selective Classifier?](https://mlsanity.com/paper/2510.20242/) (2025)
- [Evaluating Epistemic Uncertainty: Beyond OOD Detection and Active Learning](https://mlsanity.com/paper/2607.14817/) (2026)
- [On the Reliability and Stability of Selective Methods in Malware Classification Tasks](https://mlsanity.com/paper/2505.22843/) (2025)
- [Trusted Uncertainty in Large Language Models: A Unified Framework for Confidence Calibration and Risk-Controlled Refusal](https://mlsanity.com/paper/2509.01455/) (2025)
- [Leveraging Data to Say No: Memory Augmented Plug-and-Play Selective Prediction](https://mlsanity.com/paper/2601.22570/) (2026)
- [Better than Average: Spatially-Aware Aggregation of Segmentation Uncertainty Improves Downstream Performance](https://mlsanity.com/paper/2603.29941/) (2026)
- [Confidence-Gated Robot Autonomy: When Does Uncertainty Actually Help?](https://mlsanity.com/paper/2605.18045/) (2026)
- [Does Your Wildfire Prediction Model Actually Work, or Just Score Well?](https://mlsanity.com/paper/2605.18911/) (2026)
- [AnchorScore: A CLIP-Based Diagnostic of MLLM Annotation Difficulty](https://mlsanity.com/paper/2608.16690/) (2026)
- [HakemBench: A Turkish Benchmark of Typed Decisions](https://mlsanity.com/paper/2610.02293/) (2026)
- [Evaluating Medical LLMs by Levels of Autonomy: A Survey Moving from Benchmarks to Applications](https://mlsanity.com/paper/2510.17764/) (2025)
- [Filtering instances and rejecting predictions to obtain reliable models in healthcare](https://mlsanity.com/paper/2510.24368/) (2025)
- [A Systematic Analysis of Out-of-Distribution Detection Under Representation and Training Paradigm Shifts](https://mlsanity.com/paper/2511.11934/) (2025)
- [Estimation of Confidence Bounds in Binary Classification using Wilson Score Kernel Density Estimation](https://mlsanity.com/paper/2602.20947/) (2026)
- [From Misclassifications to Outliers: Joint Reliability Assessment in Classification](https://mlsanity.com/paper/2603.03903/) (2026)
- [The Confidence Gate Theorem: When Should Ranked Decision Systems Abstain?](https://mlsanity.com/paper/2603.09947/) (2026)
- [Less Data Approximates More: Earning Faithful Confidence in High-Stakes Domains](https://mlsanity.com/paper/2604.08454/) (2026)
- [CHASE: Competing Hypotheses for Ambiguity-Aware Selective Prediction](https://mlsanity.com/paper/2605.01346/) (2026)
- [A Regime Theory of Controller Class Selection for LLM Action Decisions](https://mlsanity.com/paper/2605.06339/) (2026)
- [ECUASn: A family of metrics for principled evaluation of uncertainty-augmented systems](https://mlsanity.com/paper/2605.20490/) (2026)
- [Proper Scoring Rules for Agentic Uncertainty Quantification](https://mlsanity.com/paper/2605.24756/) (2026)
- [Valid Per-Field Selective Risk Control for Document Extraction: Three Failure Modes, a Validity Ladder, and When Conditioning Pays](https://mlsanity.com/paper/2608.14639/) (2026)
- [One Score, Two Decisions: Selective Prediction on the Rare-Disease Tail](https://mlsanity.com/paper/2608.14683/) (2026)
- [ReliaGate: Reliability Routing for Low-Stakes Wearable Stress Prediction](https://mlsanity.com/paper/2608.15951/) (2026)
- [Verifiable abstention makes AI leak diagnosis accountable in urban water distribution networks](https://mlsanity.com/paper/2608.18836/) (2026)
- [Counterfactual Fragility Certificates: Exposing High-Confidence Brittleness under Structured Evidence Failure](https://mlsanity.com/paper/2609.00366/) (2026)
- [How Many Labels Does Model Choice Need? Certificates and Budgets for Selective Prediction](https://mlsanity.com/paper/2609.18622/) (2026)

## Abstract (arXiv)

Selective Classification, wherein models can reject low-confidence predictions, promises reliable translation of machine-learning based classification systems to real-world scenarios such as clinical diagnostics. While current evaluation of these systems typically assumes fixed working points based on pre-defined rejection thresholds, methodological progress requires benchmarking the general performance of systems akin to the AUROC in standard classification. In this work, we define 5 requirements for multi-threshold metrics in selective classification regarding task alignment, interpretability, and flexibility, and show how current approaches fail to meet them. We propose the Area under the Generalized Risk Coverage curve (AUGRC), which meets all requirements and can be directly interpreted as the average risk of undetected failures. We empirically demonstrate the relevance of AUGRC on a comprehensive benchmark spanning 6 data sets and 13 confidence scoring functions. We find that the proposed metric substantially changes metric rankings on 5 out of the 6 data sets.