# Through the Data Management Lens: Experimental Analysis and Evaluation of Fair Classification

arXiv: https://arxiv.org/abs/2101.07361 · HTML: https://mlsanity.com/paper/2101.07361/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 50 of its 115 references that are in the map.
- First posted on arXiv 18 Jan 2021.

## Builds on (most cited first, 30 of 50)

- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Counterfactual Fairness](https://mlsanity.com/paper/1703.06856/) (2017)
- [Certifying and removing disparate impact](https://mlsanity.com/paper/1412.3756/) (2014)
- [Fair prediction with disparate impact: A study of bias in recidivism prediction instruments](https://mlsanity.com/paper/1610.07524/) (2016)
- [Mitigating Unwanted Biases with Adversarial Learning](https://mlsanity.com/paper/1801.07593/) (2018)
- [A Reductions Approach to Fair Classification](https://mlsanity.com/paper/1803.02453/) (2018)
- [Inherent Trade-Offs in the Fair Determination of Risk Scores](https://mlsanity.com/paper/1609.05807/) (2016)
- [On Fairness and Calibration](https://mlsanity.com/paper/1709.02012/) (2017)
- [Algorithmic decision making and the cost of fairness](https://mlsanity.com/paper/1701.08230/) (2017)
- [Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness](https://mlsanity.com/paper/1711.05144/) (2017)
- [Fairness in Machine Learning: A Survey](https://mlsanity.com/paper/2010.04053/) (2020)
- [Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment](https://mlsanity.com/paper/1610.08452/) (2016)
- [Learning Adversarially Fair and Transferable Representations](https://mlsanity.com/paper/1802.06309/) (2018)
- [Fairness in Criminal Justice Risk Assessments: The State of the Art](https://mlsanity.com/paper/1703.09207/) (2017)
- [Fairness without Demographics through Adversarially Reweighted Learning](https://mlsanity.com/paper/2006.13114/) (2020)
- [The Variational Fair Autoencoder](https://mlsanity.com/paper/1511.00830/) (2015)
- [A comparative study of fairness-enhancing interventions in machine learning](https://mlsanity.com/paper/1802.04422/) (2018)
- [Avoiding Discrimination through Causal Reasoning](https://mlsanity.com/paper/1706.02744/) (2017)
- [Path-Specific Counterfactual Fairness](https://mlsanity.com/paper/1802.08139/) (2018)
- [Classification with Fairness Constraints: A Meta-Algorithm with Provable Guarantees](https://mlsanity.com/paper/1806.06055/) (2018)
- [Fair Inference On Outcomes](https://mlsanity.com/paper/1705.10378/) (2017)
- [An Intersectional Definition of Fairness](https://mlsanity.com/paper/1807.08362/) (2018)
- [Fairness Testing: Testing Software for Discrimination](https://mlsanity.com/paper/1709.03221/) (2017)
- [Ethical Machine Learning in Health Care](https://mlsanity.com/paper/2009.10576/) (2020)
- [DoWhy: An End-to-End Library for Causal Inference](https://mlsanity.com/paper/2011.04216/) (2020)
- [A Confidence-Based Approach for Balancing Fairness and Accuracy](https://mlsanity.com/paper/1601.05764/) (2016)
- [A causal framework for discovering and removing direct and indirect discrimination](https://mlsanity.com/paper/1611.07509/) (2016)
- [Learning Certified Individually Fair Representations](https://mlsanity.com/paper/2002.10312/) (2020)
- [Satisfying Real-world Goals with Dataset Constraints](https://mlsanity.com/paper/1606.07558/) (2016)

## Built on by (most cited first, 13 of 13)

- [OTClean: Data Cleaning for Conditional Independence Violations using Optimal Transport](https://mlsanity.com/paper/2403.02372/) (2024)
- [Consistent Range Approximation for Fair Predictive Modeling](https://mlsanity.com/paper/2212.10839/) (2022)
- [FairAutoML: Embracing Unfairness Mitigation in AutoML](https://mlsanity.com/paper/2111.06495/) (2021)
- [How Far Can Fairness Constraints Help Recover From Biased Data?](https://mlsanity.com/paper/2312.10396/) (2023)
- [Enforcing Conditional Independence for Fair Representation Learning and Causal Image Generation](https://mlsanity.com/paper/2404.13798/) (2024)
- [Is it Still Fair? A Comparative Evaluation of Fairness Algorithms through the Lens of Covariate Drift](https://mlsanity.com/paper/2409.12428/) (2024)
- [Non-Invasive Fairness in Learning through the Lens of Data Drift](https://mlsanity.com/paper/2303.17566/) (2023)
- [CausalPre: Scalable and Effective Data Pre-Processing for Causal Fairness](https://mlsanity.com/paper/2509.15199/) (2025)
- [On Comparing Fair Classifiers under Data Bias](https://mlsanity.com/paper/2302.05906/) (2023)
- [Adaptive Boosting with Fairness-aware Reweighting Technique for Fair Classification](https://mlsanity.com/paper/2401.03097/) (2024)
- [Stress-Testing ML Pipelines with Adversarial Data Corruption](https://mlsanity.com/paper/2506.01230/) (2025)
- [Explainable Disparity Compensation for Efficient Fair Ranking](https://mlsanity.com/paper/2307.14366/) (2023)
- [Trustworthy Data- and ML-Ops for Intelligent Transportation Systems and Logistics](https://mlsanity.com/paper/2610.01282/) (2026)

## Abstract (arXiv)

Classification, a heavily-studied data-driven machine learning task, drives an increasing number of prediction systems involving critical human decisions such as loan approval and criminal risk assessment. However, classifiers often demonstrate discriminatory behavior, especially when presented with biased data. Consequently, fairness in classification has emerged as a high-priority research area. Data management research is showing an increasing presence and interest in topics related to data and algorithmic fairness, including the topic of fair classification. The interdisciplinary efforts in fair classification, with machine learning research having the largest presence, have resulted in a large number of fairness notions and a wide range of approaches that have not been systematically evaluated and compared. In this paper, we contribute a broad analysis of 13 fair classification approaches and additional variants, over their correctness, fairness, efficiency, scalability, robustness to data errors, sensitivity to underlying ML model, data efficiency, and stability using a variety of metrics and real-world datasets. Our analysis highlights novel insights on the impact of different metrics and high-level approach characteristics on different aspects of performance. We also discuss general principles for choosing approaches suitable for different practical settings, and identify areas where data-management-centric solutions are likely to have the most impact.