# Is it Still Fair? A Comparative Evaluation of Fairness Algorithms through the Lens of Covariate Drift

arXiv: https://arxiv.org/abs/2409.12428 · HTML: https://mlsanity.com/paper/2409.12428/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 41 references that are in the map.
- First posted on arXiv 19 Sep 2024.

## Builds on (most cited first, 22 of 22)

- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [Certifying and removing disparate impact](https://mlsanity.com/paper/1412.3756/) (2014)
- [Fair prediction with disparate impact: A study of bias in recidivism prediction instruments](https://mlsanity.com/paper/1610.07524/) (2016)
- [Mitigating Unwanted Biases with Adversarial Learning](https://mlsanity.com/paper/1801.07593/) (2018)
- [On Fairness and Calibration](https://mlsanity.com/paper/1709.02012/) (2017)
- [Fairness in Criminal Justice Risk Assessments: The State of the Art](https://mlsanity.com/paper/1703.09207/) (2017)
- [A comparative study of fairness-enhancing interventions in machine learning](https://mlsanity.com/paper/1802.04422/) (2018)
- [A Unified Approach to Quantifying Algorithmic Unfairness: Measuring Individual & Group Unfairness via Inequality Indices](https://mlsanity.com/paper/1807.00787/) (2018)
- [Robust Fairness under Covariate Shift](https://mlsanity.com/paper/2010.05166/) (2020)
- [A Distributionally Robust Approach to Fair Classification](https://mlsanity.com/paper/2007.09530/) (2020)
- [Turning the Tables: Biased, Imbalanced, Dynamic Tabular Datasets for ML Evaluation](https://mlsanity.com/paper/2211.13358/) (2022)
- [Fairness Transferability Subject to Bounded Distribution Shift](https://mlsanity.com/paper/2206.00129/) (2022)
- [Does enforcing fairness mitigate biases caused by subpopulation shift?](https://mlsanity.com/paper/2011.03173/) (2020)
- [Runtime Monitoring of Dynamic Fairness Properties](https://mlsanity.com/paper/2305.04699/) (2023)
- [Fair Enough: Searching for Sufficient Measures of Fairness](https://mlsanity.com/paper/2110.13029/) (2021)
- [Through the Data Management Lens: Experimental Analysis and Evaluation of Fair Classification](https://mlsanity.com/paper/2101.07361/) (2021)
- [Graph Fairness Learning under Distribution Shifts](https://mlsanity.com/paper/2401.16784/) (2024)
- [Cross-Institutional Transfer Learning for Educational Models: Implications for Model Performance, Fairness, and Equity](https://mlsanity.com/paper/2305.00927/) (2023)
- [Understanding Fairness Surrogate Functions in Algorithmic Fairness](https://mlsanity.com/paper/2310.11211/) (2023)
- [Monitoring Algorithmic Fairness under Partial Observations](https://mlsanity.com/paper/2308.00341/) (2023)
- [Context matters for fairness -- a case study on the effect of spatial distribution shifts](https://mlsanity.com/paper/2206.11436/) (2022)

## Built on by (most cited first, 3 of 3)

- [ADVICE: Answer-Dependent Verbalized Confidence Estimation](https://mlsanity.com/paper/2510.10913/) (2025)
- [Evaluating Model Retraining under Drift: Paired Comparisons of Cumulative Subgroup Disparity](https://mlsanity.com/paper/2609.09788/) (2026)
- [The Mirage of Calibrated Confidence: Trajectory-Independence of Verbalized Confidence in Vision-Language Models](https://mlsanity.com/paper/2609.18453/) (2026)

## Abstract (arXiv)

Over the last few decades, machine learning (ML) applications have grown exponentially, yielding several benefits to society. However, these benefits are tempered with concerns of discriminatory behaviours exhibited by ML models. In this regard, fairness in machine learning has emerged as a priority research area. Consequently, several fairness metrics and algorithms have been developed to mitigate against discriminatory behaviours that ML models may possess. Yet still, very little attention has been paid to the problem of naturally occurring changes in data patterns (textitaka data distributional drift), and its impact on fairness algorithms and metrics. In this work, we study this problem comprehensively by analyzing 4 fairness-unaware baseline algorithms and 7 fairness-aware algorithms, carefully curated to cover the breadth of its typology, across 5 datasets including public and proprietary data, and evaluated them using 3 predictive performance and 10 fairness metrics. In doing so, we show that (1) data distributional drift is not a trivial occurrence, and in several cases can lead to serious deterioration of fairness in so-called fair models; (2) contrary to some existing literature, the size and direction of data distributional drift is not correlated to the resulting size and direction of unfairness; and (3) choice of, and training of fairness algorithms is impacted by the effect of data distributional drift which is largely ignored in the literature. Emanating from our findings, we synthesize several policy implications of data distributional drift on fairness algorithms that can be very relevant to stakeholders and practitioners.