# A Clarification of the Nuances in the Fairness Metrics Landscape

arXiv: https://arxiv.org/abs/2106.00467 · HTML: https://mlsanity.com/paper/2106.00467/

## Key facts

- Cited by 98 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 107 references that are in the map.
- First posted on arXiv 1 Jun 2021.

## Builds on (most cited first, 30 of 36)

- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [A Survey on Bias and Fairness in Machine Learning](https://mlsanity.com/paper/1908.09635/) (2019)
- [A Survey Of Methods For Explaining Black Box Models](https://mlsanity.com/paper/1802.01933/) (2018)
- [Counterfactual Fairness](https://mlsanity.com/paper/1703.06856/) (2017)
- [Certifying and removing disparate impact](https://mlsanity.com/paper/1412.3756/) (2014)
- [Fair prediction with disparate impact: A study of bias in recidivism prediction instruments](https://mlsanity.com/paper/1610.07524/) (2016)
- [Inherent Trade-Offs in the Fair Determination of Risk Scores](https://mlsanity.com/paper/1609.05807/) (2016)
- [Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness](https://mlsanity.com/paper/1711.05144/) (2017)
- [Fairness in Machine Learning: A Survey](https://mlsanity.com/paper/2010.04053/) (2020)
- [Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment](https://mlsanity.com/paper/1610.08452/) (2016)
- [Fairness in Criminal Justice Risk Assessments: The State of the Art](https://mlsanity.com/paper/1703.09207/) (2017)
- [Causal Discovery with Continuous Additive Noise Models](https://mlsanity.com/paper/1309.6779/) (2013)
- [A Survey on the Explainability of Supervised Machine Learning](https://mlsanity.com/paper/2011.07876/) (2020)
- [The Variational Fair Autoencoder](https://mlsanity.com/paper/1511.00830/) (2015)
- [Avoiding Discrimination through Causal Reasoning](https://mlsanity.com/paper/1706.02744/) (2017)
- [Fairness in Machine Learning](https://mlsanity.com/paper/2012.15816/) (2020)
- [Delayed Impact of Fair Machine Learning](https://mlsanity.com/paper/1803.04383/) (2018)
- [Path-Specific Counterfactual Fairness](https://mlsanity.com/paper/1802.08139/) (2018)
- [Fair Regression: Quantitative Definitions and Reduction-based Algorithms](https://mlsanity.com/paper/1905.12843/) (2019)
- [A Convex Framework for Fair Regression](https://mlsanity.com/paper/1706.02409/) (2017)
- [The Frontiers of Fairness in Machine Learning](https://mlsanity.com/paper/1810.08810/) (2018)
- [Wasserstein Fair Classification](https://mlsanity.com/paper/1907.12059/) (2019)
- [On the Apparent Conflict Between Individual and Group Fairness](https://mlsanity.com/paper/1912.06883/) (2019)
- [Fairness in Reinforcement Learning](https://mlsanity.com/paper/1611.03071/) (2016)
- [An Empirical Study of Rich Subgroup Fairness for Machine Learning](https://mlsanity.com/paper/1808.08166/) (2018)
- [Metric Learning for Individual Fairness](https://mlsanity.com/paper/1906.00250/) (2019)
- [FlipTest: Fairness Testing via Optimal Transport](https://mlsanity.com/paper/1906.09218/) (2019)
- [A Short-term Intervention for Long-term Fairness in the Labor Market](https://mlsanity.com/paper/1712.00064/) (2017)
- [A Survey of Learning Causality with Data: Problems and Methods](https://mlsanity.com/paper/1809.09337/) (2018)
- [On the Moral Justification of Statistical Parity](https://mlsanity.com/paper/2011.02079/) (2020)

## Built on by (most cited first, 30 of 98)

- [Generative models improve fairness of medical classifiers under distribution shifts](https://mlsanity.com/paper/2304.09218/) (2023)
- [MEDFAIR: Benchmarking Fairness for Medical Imaging](https://mlsanity.com/paper/2210.01725/) (2022)
- [Selective Regression Under Fairness Criteria](https://mlsanity.com/paper/2110.15403/) (2021)
- [A Comprehensive Survey on Trustworthy Recommender Systems](https://mlsanity.com/paper/2209.10117/) (2022)
- [On the Independence of Association Bias and Empirical Fairness in Language Models](https://mlsanity.com/paper/2304.10153/) (2023)
- [Fairness in Recommendation: Foundations, Methods and Applications](https://mlsanity.com/paper/2205.13619/) (2022)
- [Bias patterns in the application of LLMs for clinical decision support: A comprehensive study](https://mlsanity.com/paper/2404.15149/) (2024)
- [Challenges in Applying Explainability Methods to Improve the Fairness of NLP Models](https://mlsanity.com/paper/2206.03945/) (2022)
- [Conformalized Fairness via Quantile Regression](https://mlsanity.com/paper/2210.02015/) (2022)
- [TSGM: A Flexible Framework for Generative Modeling of Synthetic Time Series](https://mlsanity.com/paper/2305.11567/) (2023)
- [A Critical Survey on Fairness Benefits of Explainable AI](https://mlsanity.com/paper/2310.13007/) (2023)
- [Uncertainty-based Fairness Measures](https://mlsanity.com/paper/2312.11299/) (2023)
- [Fairness and Bias Mitigation in Computer Vision: A Survey](https://mlsanity.com/paper/2408.02464/) (2024)
- [FairIF: Boosting Fairness in Deep Learning via Influence Functions with Validation Set Sensitive Attributes](https://mlsanity.com/paper/2201.05759/) (2022)
- [Being Right for Whose Right Reasons?](https://mlsanity.com/paper/2306.00639/) (2023)
- [Bias-inducing geometries: an exactly solvable data model with fairness implications](https://mlsanity.com/paper/2205.15935/) (2022)
- [An Empirical Analysis of Fairness Notions under Differential Privacy](https://mlsanity.com/paper/2302.02910/) (2023)
- [Unmasking Bias in AI: A Systematic Review of Bias Detection and Mitigation Strategies in Electronic Health Record-based Models](https://mlsanity.com/paper/2310.19917/) (2023)
- [FairPFN: A Tabular Foundation Model for Causal Fairness](https://mlsanity.com/paper/2506.07049/) (2025)
- [FairPFN: Transformers Can do Counterfactual Fairness](https://mlsanity.com/paper/2407.05732/) (2024)
- [What Constitutes a Less Discriminatory Algorithm?](https://mlsanity.com/paper/2412.18138/) (2024)
- [Is Your Model "MADD"? A Novel Metric to Evaluate Algorithmic Fairness for Predictive Student Models](https://mlsanity.com/paper/2305.15342/) (2023)
- [Properties of fairness measures in the context of varying class imbalance and protected group ratios](https://mlsanity.com/paper/2411.08425/) (2024)
- [Towards Procedural Fairness: Uncovering Biases in How a Toxic Language Classifier Uses Sentiment Information](https://mlsanity.com/paper/2210.10689/) (2022)
- [Improving LLM Group Fairness on Tabular Data via In-Context Learning](https://mlsanity.com/paper/2412.04642/) (2024)
- [Error Parity Fairness: Testing for Group Fairness in Regression Tasks](https://mlsanity.com/paper/2208.08279/) (2022)
- [Algorithmic decision making methods for fair credit scoring](https://mlsanity.com/paper/2209.07912/) (2022)
- [BIASeD: Bringing Irrationality into Automated System Design](https://mlsanity.com/paper/2210.01122/) (2022)
- [Fairness Hacking: The Malicious Practice of Shrouding Unfairness in Algorithms](https://mlsanity.com/paper/2311.06826/) (2023)
- [What is Fair? Defining Fairness in Machine Learning for Health](https://mlsanity.com/paper/2406.09307/) (2024)

## Abstract (arXiv)

In recent years, the problem of addressing fairness in Machine Learning (ML) and automatic decision-making has attracted a lot of attention in the scientific communities dealing with Artificial Intelligence. A plethora of different definitions of fairness in ML have been proposed, that consider different notions of what is a "fair decision" in situations impacting individuals in the population. The precise differences, implications and "orthogonality" between these notions have not yet been fully analyzed in the literature. In this work, we try to make some order out of this zoo of definitions.