# Selective Classification Can Magnify Disparities Across Groups

arXiv: https://arxiv.org/abs/2010.14134 · HTML: https://mlsanity.com/paper/2010.14134/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Oct 2020.

## Built on by (most cited first, 30 of 36)

- [D2C: Diffusion-Denoising Models for Few-shot Conditional Generation](https://mlsanity.com/paper/2106.06819/) (2021)
- [Llamas Know What GPTs Don't Show: Surrogate Models for Confidence Estimation](https://mlsanity.com/paper/2311.08877/) (2023)
- [Selective Regression Under Fairness Criteria](https://mlsanity.com/paper/2110.15403/) (2021)
- [Calibrated ensembles can mitigate accuracy tradeoffs under distribution shift](https://mlsanity.com/paper/2207.08977/) (2022)
- [Calibrated Selective Classification](https://mlsanity.com/paper/2208.12084/) (2022)
- [Classification with abstention but without disparities](https://mlsanity.com/paper/2102.12258/) (2021)
- [Towards Better Selective Classification](https://mlsanity.com/paper/2206.09034/) (2022)
- [Learning to Defer with Limited Expert Predictions](https://mlsanity.com/paper/2304.07306/) (2023)
- [Selective Ensembles for Consistent Predictions](https://mlsanity.com/paper/2111.08230/) (2021)
- [Teacher's pet: understanding and mitigating biases in distillation](https://mlsanity.com/paper/2106.10494/) (2021)
- [Reasoning over Public and Private Data in Retrieval-Based Systems](https://mlsanity.com/paper/2203.11027/) (2022)
- [Human-AI Collaboration in Decision-Making: Beyond Learning to Defer](https://mlsanity.com/paper/2206.13202/) (2022)
- [Plugin estimators for selective classification with out-of-distribution detection](https://mlsanity.com/paper/2301.12386/) (2023)
- [The challenge of uncertainty quantification of large language models in medicine](https://mlsanity.com/paper/2504.05278/) (2025)
- [AUC-based Selective Classification](https://mlsanity.com/paper/2210.10703/) (2022)
- [Modulating Language Model Experiences through Frictions](https://mlsanity.com/paper/2407.12804/) (2024)
- [Cross-Institutional Transfer Learning for Educational Models: Implications for Model Performance, Fairness, and Equity](https://mlsanity.com/paper/2305.00927/) (2023)
- [Fair Classifiers that Abstain without Harm](https://mlsanity.com/paper/2310.06205/) (2023)
- [Quantifying and mitigating the impact of label errors on model disparity metrics](https://mlsanity.com/paper/2310.02533/) (2023)
- [AI, Meet Human: Learning Paradigms for Hybrid Decision Making Systems](https://mlsanity.com/paper/2402.06287/) (2024)
- [FairRoad: Achieving Fairness for Recommender Systems with Optimized Antidote Data](https://mlsanity.com/paper/2212.06750/) (2022)
- [Training Private Models That Know What They Don't Know](https://mlsanity.com/paper/2305.18393/) (2023)
- [A Unifying Post-Processing Framework for Multi-Objective Learn-to-Defer Problems](https://mlsanity.com/paper/2407.12710/) (2024)
- [Classification with Conceptual Safeguards](https://mlsanity.com/paper/2411.04342/) (2024)
- [Improving Predictor Reliability with Selective Recalibration](https://mlsanity.com/paper/2410.05407/) (2024)
- [Operationalizing the Blueprint for an AI Bill of Rights: Recommendations for Practitioners, Researchers, and Policy Makers](https://mlsanity.com/paper/2407.08689/) (2024)
- [Interpretable and Fair Mechanisms for Abstaining Classifiers](https://mlsanity.com/paper/2503.18826/) (2025)
- [Confidential Guardian: Cryptographically Prohibiting the Abuse of Model Abstention](https://mlsanity.com/paper/2505.23968/) (2025)
- [What Does It Take to Build a Performant Selective Classifier?](https://mlsanity.com/paper/2510.20242/) (2025)
- [Interviewer-Candidate Role Play: Towards Developing Real-World NLP Systems](https://mlsanity.com/paper/2107.00315/) (2021)

## Abstract (arXiv)

Selective classification, in which models can abstain on uncertain predictions, is a natural approach to improving accuracy in settings where errors are costly but abstentions are manageable. In this paper, we find that while selective classification can improve average accuracies, it can simultaneously magnify existing accuracy disparities between various groups within a population, especially in the presence of spurious correlations. We observe this behavior consistently across five vision and NLP datasets. Surprisingly, increasing abstentions can even decrease accuracies on some groups. To better understand this phenomenon, we study the margin distribution, which captures the model's confidences over all predictions. For symmetric margin distributions, we prove that whether selective classification monotonically improves or worsens accuracy is fully determined by the accuracy at full coverage (i.e., without any abstentions) and whether the distribution satisfies a property we call left-log-concavity. Our analysis also shows that selective classification tends to magnify full-coverage accuracy disparities. Motivated by our analysis, we train distributionally-robust models that achieve similar full-coverage accuracies across groups and show that selective classification uniformly improves each group on these models. Altogether, our results suggest that selective classification should be used with care and underscore the importance of training models to perform equally well across groups at full coverage.