# On the relation between accuracy and fairness in binary classification

arXiv: https://arxiv.org/abs/1505.05723 · HTML: https://mlsanity.com/paper/1505.05723/

## Key facts

- Cited by 47 papers in the map (data through 7 Oct 2026).
- First posted on arXiv 21 May 2015.

## Built on by (most cited first, 30 of 47)

- [A survey on datasets for fairness-aware machine learning](https://mlsanity.com/paper/2110.00530/) (2021)
- [Perturbation Augmentation for Fairer NLP](https://mlsanity.com/paper/2205.12586/) (2022)
- [Understanding and Mitigating Annotation Bias in Facial Expression Recognition](https://mlsanity.com/paper/2108.08504/) (2021)
- [The Road to Explainability is Paved with Bias: Measuring the Fairness of Explanations](https://mlsanity.com/paper/2205.03295/) (2022)
- [Understanding and Improving Fairness-Accuracy Trade-offs in Multi-Task Learning](https://mlsanity.com/paper/2106.02705/) (2021)
- [Evaluating Fairness of Machine Learning Models Under Uncertain and Incomplete Information](https://mlsanity.com/paper/2102.08410/) (2021)
- [Fairness in Ranking under Uncertainty](https://mlsanity.com/paper/2107.06720/) (2021)
- [Fair Machine Unlearning: Data Removal while Mitigating Disparities](https://mlsanity.com/paper/2307.14754/) (2023)
- [Characterizing the risk of fairwashing](https://mlsanity.com/paper/2106.07504/) (2021)
- [Counterfactual Fairness with Partially Known Causal Graph](https://mlsanity.com/paper/2205.13972/) (2022)
- [A Framework for Fairness: A Systematic Review of Existing Fair AI Solutions](https://mlsanity.com/paper/2112.05700/) (2021)
- [Algorithmic Fairness Verification with Graphical Models](https://mlsanity.com/paper/2109.09447/) (2021)
- [Counterfactual Fairness Is Basically Demographic Parity](https://mlsanity.com/paper/2208.03843/) (2022)
- [Disentangled Representations from Non-Disentangled Models](https://mlsanity.com/paper/2102.06204/) (2021)
- [The Pursuit of Fairness in Artificial Intelligence Models: A Survey](https://mlsanity.com/paper/2403.17333/) (2024)
- [How Spurious Features Are Memorized: Precise Analysis for Random and NTK Features](https://mlsanity.com/paper/2305.12100/) (2023)
- [To the Fairness Frontier and Beyond: Identifying, Quantifying, and Optimizing the Fairness-Accuracy Pareto Frontier](https://mlsanity.com/paper/2206.00074/) (2022)
- [Group Meritocratic Fairness in Linear Contextual Bandits](https://mlsanity.com/paper/2206.03150/) (2022)
- [Survey on Fairness Notions and Related Tensions](https://mlsanity.com/paper/2209.13012/) (2022)
- [Spurious Correlations in High Dimensional Regression: The Roles of Regularization, Simplicity Bias and Over-Parameterization](https://mlsanity.com/paper/2502.01347/) (2025)
- [An Empirical Comparison of Bias Reduction Methods on Real-World Problems in High-Stakes Policy Settings](https://mlsanity.com/paper/2105.06442/) (2021)
- [Costs and Benefits of Fair Regression](https://mlsanity.com/paper/2106.08812/) (2021)
- [How Biased are Your Features?: Computing Fairness Influence Functions with Global Sensitivity Analysis](https://mlsanity.com/paper/2206.00667/) (2022)
- [Interventional Fairness on Partially Known Causal Graphs: A Constrained Optimization Approach](https://mlsanity.com/paper/2401.10632/) (2024)
- [Fair Sparse Regression with Clustering: An Invex Relaxation for a Combinatorial Problem](https://mlsanity.com/paper/2102.09704/) (2021)
- [Evaluation of group fairness measures in student performance prediction problems](https://mlsanity.com/paper/2208.10625/) (2022)
- [Overcoming Fairness Trade-offs via Pre-processing: A Causal Perspective](https://mlsanity.com/paper/2501.14710/) (2025)
- [Fairness in Forecasting of Observations of Linear Dynamical Systems](https://mlsanity.com/paper/2209.05274/) (2022)
- [A Review of Fairness and A Practical Guide to Selecting Context-Appropriate Fairness Metrics in Machine Learning](https://mlsanity.com/paper/2411.06624/) (2024)
- [Causality Is Key to Understand and Balance Multiple Goals in Trustworthy ML and Foundation Models](https://mlsanity.com/paper/2502.21123/) (2025)

## Abstract (arXiv)

Our study revisits the problem of accuracy-fairness tradeoff in binary classification. We argue that comparison of non-discriminatory classifiers needs to account for different rates of positive predictions, otherwise conclusions about performance may be misleading, because accuracy and discrimination of naive baselines on the same dataset vary with different rates of positive predictions. We provide methodological recommendations for sound comparison of non-discriminatory classifiers, and present a brief theoretical and empirical analysis of tradeoffs between accuracy and non-discrimination.