# Evaluating Fairness Metrics in the Presence of Dataset Bias

arXiv: https://arxiv.org/abs/1809.09245 · HTML: https://mlsanity.com/paper/1809.09245/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Sep 2018.

## Built on by (most cited first, 15 of 15)

- [Evaluating the Fairness of Deep Learning Uncertainty Estimates in Medical Image Analysis](https://mlsanity.com/paper/2303.03242/) (2023)
- [Fair Enough: Searching for Sufficient Measures of Fairness](https://mlsanity.com/paper/2110.13029/) (2021)
- [Fairness Score and Process Standardization: Framework for Fairness Certification in Artificial Intelligence Systems](https://mlsanity.com/paper/2201.06952/) (2022)
- [Do Deep Networks Transfer Invariances Across Classes?](https://mlsanity.com/paper/2203.09739/) (2022)
- [Low-Cost Algorithmic Recourse for Users With Uncertain Cost Functions](https://mlsanity.com/paper/2111.01235/) (2021)
- [Fairness and Bias in Robot Learning](https://mlsanity.com/paper/2207.03444/) (2022)
- [QueerBench: Quantifying Discrimination in Language Models Toward Queer Identities](https://mlsanity.com/paper/2406.12399/) (2024)
- [Data AUDIT: Identifying Attribute Utility- and Detectability-Induced Bias in Task Models](https://mlsanity.com/paper/2304.03218/) (2023)
- [Measure Twice, Cut Once: Quantifying Bias and Fairness in Deep Neural Networks](https://mlsanity.com/paper/2110.04397/) (2021)
- [A Seven-Layer Model for Standardising AI Fairness Assessment](https://mlsanity.com/paper/2212.11207/) (2022)
- [QueerGen: How LLMs Reflect Societal Norms on Gender and Sexuality in Sentence Completion Tasks](https://mlsanity.com/paper/2601.20731/) (2026)
- [Navigating Fairness in Radiology AI: Concepts, Consequences,and Crucial Considerations](https://mlsanity.com/paper/2306.01333/) (2023)
- [Generalizing Fairness to Generative Language Models via Reformulation of Non-discrimination Criteria](https://mlsanity.com/paper/2403.08564/) (2024)
- [Detecting Dataset Bias in Medical AI: A Generalized and Modality-Agnostic Auditing Framework](https://mlsanity.com/paper/2503.09969/) (2025)
- [Auditing and Mitigating Bias in Gender Classification Algorithms: A Data-Centric Approach](https://mlsanity.com/paper/2510.17873/) (2025)

## Abstract (arXiv)

Data-driven algorithms play a large role in decision making across a variety of industries. Increasingly, these algorithms are being used to make decisions that have significant ramifications for people's social and economic well-being, e.g. in sentencing, loan approval, and policing. Amid the proliferation of such systems there is a growing concern about their potential discriminatory impact. In particular, machine learning systems which are trained on biased data have the potential to learn and perpetuate those biases. A central challenge for practitioners is thus to determine whether their models display discriminatory bias. Here we present a case study in which we frame the issue of bias detection as a causal inference problem with observational data. We enumerate two main causes of bias, sampling bias and label bias, and we investigate the abilities of six different fairness metrics to detect each bias type. Based on these investigations, we propose a set of best practice guidelines to select the fairness metric that is most likely to detect bias if it is present. Additionally, we aim to identify the conditions in which certain fairness metrics may fail to detect bias and instead give practitioners a false belief that their biased model is making fair decisions.