# Why Is My Classifier Discriminatory?

arXiv: https://arxiv.org/abs/1805.12002 · HTML: https://mlsanity.com/paper/1805.12002/

## Key facts

- Cited by 143 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 May 2018.

## Built on by (most cited first, 30 of 143)

- [Trustworthy AI: From Principles to Practices](https://mlsanity.com/paper/2110.01167/) (2021)
- [Evaluating Deep Neural Networks Trained on Clinical Images in Dermatology with the Fitzpatrick 17k Dataset](https://mlsanity.com/paper/2104.09957/) (2021)
- [Bias Mitigation for Machine Learning Classifiers: A Comprehensive Survey](https://mlsanity.com/paper/2207.07068/) (2022)
- [Data Collection and Quality Challenges in Deep Learning: A Data-Centric AI Perspective](https://mlsanity.com/paper/2112.06409/) (2021)
- [Long-Tailed Recognition via Weight Balancing](https://mlsanity.com/paper/2203.14197/) (2022)
- [Trustworthy AI: A Computational Perspective](https://mlsanity.com/paper/2107.06641/) (2021)
- [MEDFAIR: Benchmarking Fairness for Medical Imaging](https://mlsanity.com/paper/2210.01725/) (2022)
- [The Spotlight: A General Method for Discovering Systematic Errors in Deep Learning Models](https://mlsanity.com/paper/2107.00758/) (2021)
- [Multi-group Agnostic PAC Learnability](https://mlsanity.com/paper/2105.09989/) (2021)
- [The Road to Explainability is Paved with Bias: Measuring the Fairness of Explanations](https://mlsanity.com/paper/2205.03295/) (2022)
- [Achieving Fairness at No Utility Cost via Data Reweighing with Influence](https://mlsanity.com/paper/2202.00787/) (2022)
- [Representation Bias in Data: A Survey on Identification and Resolution Techniques](https://mlsanity.com/paper/2203.11852/) (2022)
- [Towards Transparency in Dermatology Image Datasets with Skin Tone Annotations by Experts, Crowds, and an Algorithm](https://mlsanity.com/paper/2207.02942/) (2022)
- [CLIP the Bias: How Useful is Balancing Data in Multimodal Learning?](https://mlsanity.com/paper/2403.04547/) (2024)
- [Emergent Unfairness in Algorithmic Fairness-Accuracy Trade-Off Research](https://mlsanity.com/paper/2102.01203/) (2021)
- [Arbitrariness and Social Prediction: The Confounding Role of Variance in Fair Classification](https://mlsanity.com/paper/2301.11562/) (2023)
- [Most Influential Subset Selection: Challenges, Promises, and Beyond](https://mlsanity.com/paper/2409.18153/) (2024)
- [Representation Matters: Assessing the Importance of Subgroup Allocations in Training Data](https://mlsanity.com/paper/2103.03399/) (2021)
- [Repairing Neural Networks by Leaving the Right Past Behind](https://mlsanity.com/paper/2207.04806/) (2022)
- [Beyond Confidence: Reliable Models Should Also Consider Atypicality](https://mlsanity.com/paper/2305.18262/) (2023)
- [Fairness without Imputation: A Decision Tree Approach for Fair Prediction with Missing Values](https://mlsanity.com/paper/2109.10431/) (2021)
- [FairTune: Optimizing Parameter Efficient Fine Tuning for Fairness in Medical Image Analysis](https://mlsanity.com/paper/2310.05055/) (2023)
- [Algorithm Fairness in AI for Medicine and Healthcare](https://mlsanity.com/paper/2110.00603/) (2021)
- [Certifying Robustness to Programmable Data Bias in Decision Trees](https://mlsanity.com/paper/2110.04363/) (2021)
- [Hyperparameter Optimization Is Deceiving Us, and How to Stop It](https://mlsanity.com/paper/2102.03034/) (2021)
- [Write It Like You See It: Detectable Differences in Clinical Notes By Race Lead To Differential Model Recommendations](https://mlsanity.com/paper/2205.03931/) (2022)
- [Counterfactual Fairness with Partially Known Causal Graph](https://mlsanity.com/paper/2205.13972/) (2022)
- [Aleatoric and Epistemic Discrimination: Fundamental Limits of Fairness Interventions](https://mlsanity.com/paper/2301.11781/) (2023)
- [Understanding and Mitigating Accuracy Disparity in Regression](https://mlsanity.com/paper/2102.12013/) (2021)
- [The Dataset Multiplicity Problem: How Unreliable Data Impacts Predictions](https://mlsanity.com/paper/2304.10655/) (2023)

## Abstract (arXiv)

Recent attempts to achieve fairness in predictive models focus on the balance between fairness and accuracy. In sensitive applications such as healthcare or criminal justice, this trade-off is often undesirable as any increase in prediction error could have devastating consequences. In this work, we argue that the fairness of predictions should be evaluated in context of the data, and that unfairness induced by inadequate samples sizes or unmeasured predictive variables should be addressed through data collection, rather than by constraining the model. We decompose cost-based metrics of discrimination into bias, variance, and noise, and propose actions aimed at estimating and reducing each term. Finally, we perform case-studies on prediction of income, mortality, and review ratings, confirming the value of this analysis. We find that data collection is often a means to reduce discrimination without sacrificing accuracy.