# Fairness Metrics: A Comparative Analysis

arXiv: https://arxiv.org/abs/2001.07864 · HTML: https://mlsanity.com/paper/2001.07864/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Jan 2020.

## Built on by (most cited first, 30 of 45)

- [Fairness-aware Class Imbalanced Learning](https://mlsanity.com/paper/2109.10444/) (2021)
- [Enforcing Group Fairness in Algorithmic Decision Making: Utility Maximization Under Sufficiency](https://mlsanity.com/paper/2206.02237/) (2022)
- [FairCal: Fairness Calibration for Face Verification](https://mlsanity.com/paper/2106.03761/) (2021)
- [Uncertainty-based Fairness Measures](https://mlsanity.com/paper/2312.11299/) (2023)
- [Bias in Machine Learning Models Can Be Significantly Mitigated by Careful Training: Evidence from Neuroimaging Studies](https://mlsanity.com/paper/2205.13421/) (2022)
- [Mitigating Gender Bias in Face Recognition Using the von Mises-Fisher Mixture Model](https://mlsanity.com/paper/2210.13664/) (2022)
- [FAIRER: Fairness as Decision Rationale Alignment](https://mlsanity.com/paper/2306.15299/) (2023)
- [Can Machine Learning Assist in Diagnosis of Primary Immune Thrombocytopenia? A feasibility study](https://mlsanity.com/paper/2405.20562/) (2024)
- [Improving Large Language Model (LLM) fidelity through context-aware grounding: A systematic approach to reliability and veracity](https://mlsanity.com/paper/2408.04023/) (2024)
- [How Biased are Your Features?: Computing Fairness Influence Functions with Global Sensitivity Analysis](https://mlsanity.com/paper/2206.00667/) (2022)
- [On Prediction-Modelers and Decision-Makers: Why Fairness Requires More Than a Fair Prediction Model](https://mlsanity.com/paper/2310.05598/) (2023)
- [Mining bias-target Alignment from Voronoi Cells](https://mlsanity.com/paper/2305.03691/) (2023)
- [X Hacking: The Threat of Misguided AutoML](https://mlsanity.com/paper/2401.08513/) (2024)
- [Using Self-supervised Learning Can Improve Model Fairness](https://mlsanity.com/paper/2406.02361/) (2024)
- [Understanding the Interplay of Scale, Data, and Bias in Language Models: A Case Study with BERT](https://mlsanity.com/paper/2407.21058/) (2024)
- [Retiring DeltaDP: New Distribution-Level Metrics for Demographic Parity](https://mlsanity.com/paper/2301.13443/) (2023)
- [Multi-Objective Population Based Training](https://mlsanity.com/paper/2306.01436/) (2023)
- [Evaluating the fairness of fine-tuning strategies in self-supervised learning](https://mlsanity.com/paper/2110.00538/) (2021)
- [Impact Of Missing Data Imputation On The Fairness And Accuracy Of Graph Node Classifiers](https://mlsanity.com/paper/2211.00783/) (2022)
- [On the Maximal Local Disparity of Fairness-Aware Classifiers](https://mlsanity.com/paper/2406.03255/) (2024)
- [A Review of Fairness and A Practical Guide to Selecting Context-Appropriate Fairness Metrics in Machine Learning](https://mlsanity.com/paper/2411.06624/) (2024)
- [Gender Fairness of Machine Learning Algorithms for Pain Detection](https://mlsanity.com/paper/2506.11132/) (2025)
- [Finding Representative Group Fairness Metrics Using Correlation Estimations](https://mlsanity.com/paper/2109.05697/) (2021)
- [There is an elephant in the room: Towards a critique on the use of fairness in biometrics](https://mlsanity.com/paper/2112.11193/) (2021)
- [Gender Bias in Transformer Models: A comprehensive survey](https://mlsanity.com/paper/2306.10530/) (2023)
- [Initial Exploration of Zero-Shot Privacy Utility Tradeoffs in Tabular Data Using GPT-4](https://mlsanity.com/paper/2404.05047/) (2024)
- [Outlier Detection Bias Busted: Understanding Sources of Algorithmic Bias through Data-centric Factors](https://mlsanity.com/paper/2408.13667/) (2024)
- [The Disparate Benefits of Deep Ensembles](https://mlsanity.com/paper/2410.13831/) (2024)
- [Simultaneous Improvement of ML Model Fairness and Performance by Identifying Bias in Data](https://mlsanity.com/paper/2210.13182/) (2022)
- [Identifying Reasons for Bias: An Argumentation-Based Approach](https://mlsanity.com/paper/2310.16506/) (2023)

## Abstract (arXiv)

Algorithmic fairness is receiving significant attention in the academic and broader literature due to the increasing use of predictive algorithms, including those based on artificial intelligence. One benefit of this trend is that algorithm designers and users have a growing set of fairness measures to choose from. However, this choice comes with the challenge of identifying how the different fairness measures relate to one another, as well as the extent to which they are compatible or mutually exclusive. We describe some of the most widely used fairness metrics using a common mathematical framework and present new results on the relationships among them. The results presented herein can help place both specialists and non-specialists in a better position to identify the metric best suited for their application and goals.