mlsanityData to 7 Oct 2026

A Clarification of the Nuances in the Fairness Metrics Landscape

arXiv 2106.00467 ↗First posted 1 Jun 2021cs.LG cs.CY stat.MLcited by 98

Not on the map. Why?

Abstract (arXiv)

In recent years, the problem of addressing fairness in Machine Learning (ML) and automatic decision-making has attracted a lot of attention in the scientific communities dealing with Artificial Intelligence. A plethora of different definitions of fairness in ML have been proposed, that consider different notions of what is a "fair decision" in situations impacting individuals in the population. The precise differences, implications and "orthogonality" between these notions have not yet been fully analyzed in the literature. In this work, we try to make some order out of this zoo of definitions.

Key facts. Cited by 98 papers in the map (data through 7 Oct 2026). Builds on 36 of its 107 references that are in the map. First posted on arXiv 1 Jun 2021.

Builds on

36 of 107 references.

References · 36

  1. 2016Equality of Opportunity in Supervised Learningcited by 2,000
  2. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,911
  3. 2018A Survey Of Methods For Explaining Black Box Modelscited by 970
  4. 2017Counterfactual Fairnesscited by 802
  5. 2014Certifying and removing disparate impactcited by 715
  6. 2016Fair prediction with disparate impact: A study of bias in recidivism prediction instrumentscited by 671
  7. 2016Inherent Trade-Offs in the Fair Determination of Risk Scorescited by 582
  8. 2017Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairnesscited by 381
  9. 2020Fairness in Machine Learning: A Surveycited by 369
  10. 2016Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatmentcited by 368
  11. 2017Fairness in Criminal Justice Risk Assessments: The State of the Artcited by 315
  12. 2013Causal Discovery with Continuous Additive Noise Modelscited by 252
24 more
  1. 2020A Survey on the Explainability of Supervised Machine Learningcited by 225
  2. 2015The Variational Fair Autoencodercited by 223
  3. 2020Fairness in Machine Learningcited by 194
  4. 2017Avoiding Discrimination through Causal Reasoningcited by 191
  5. 2018Delayed Impact of Fair Machine Learningcited by 183
  6. 2018Path-Specific Counterfactual Fairnesscited by 179
  7. 2019Fair Regression: Quantitative Definitions and Reduction-based Algorithmscited by 168
  8. 2017A Convex Framework for Fair Regressioncited by 147
  9. 2018The Frontiers of Fairness in Machine Learningcited by 139
  10. 2019Wasserstein Fair Classificationcited by 131
  11. 2019On the Apparent Conflict Between Individual and Group Fairnesscited by 126
  12. 2016Fairness in Reinforcement Learningcited by 93
  13. 2018An Empirical Study of Rich Subgroup Fairness for Machine Learningcited by 81
  14. 2019Metric Learning for Individual Fairnesscited by 51
  15. 2019FlipTest: Fairness Testing via Optimal Transportcited by 51
  16. 2017A Short-term Intervention for Long-term Fairness in the Labor Marketcited by 44
  17. 2018A Survey of Learning Causality with Data: Problems and Methodscited by 37
  18. 2020On the Moral Justification of Statistical Paritycited by 30
  19. 2019This Thing Called Fairness: Disciplinary Confusion Realizing a Value in Technologycited by 24
  20. 2017Provably Fair Representationscited by 22
  21. 2021Group Fairness: Independence Revisitedcited by 20
  22. 2020Review of The Book of Why: The New Science of Cause and Effectcited by 19
  23. 2019An Algorithmic Framework for Fairness Elicitationcited by 15
  24. 2021BeFair: Addressing Fairness in the Banking Sectorcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 15; 2023: 30; 2024: 30; 2025: 16; 2026: 6.

98 papers.

Sort:

By area:

  1. 2023Generative models improve fairness of medical classifiers under distribution shiftscited by 90
  2. 2022MEDFAIR: Benchmarking Fairness for Medical Imagingcited by 66
  3. 2021Selective Regression Under Fairness Criteriacited by 34
  4. 2022A Comprehensive Survey on Trustworthy Recommender Systemscited by 32
  5. 2023On the Independence of Association Bias and Empirical Fairness in Language Modelscited by 29
  6. 2022Fairness in Recommendation: Foundations, Methods and Applicationscited by 27
  7. 2024Bias patterns in the application of LLMs for clinical decision support: A comprehensive studycited by 27
  8. 2022Challenges in Applying Explainability Methods to Improve the Fairness of NLP Modelscited by 20
  9. 2022Conformalized Fairness via Quantile Regressioncited by 19
  10. 2023TSGM: A Flexible Framework for Generative Modeling of Synthetic Time Seriescited by 16
  11. 2023A Critical Survey on Fairness Benefits of Explainable AIcited by 16
  12. 2023Uncertainty-based Fairness Measurescited by 15
38 more
  1. 2022FairIF: Boosting Fairness in Deep Learning via Influence Functions with Validation Set Sensitive Attributescited by 12
  2. 2023Being Right for Whose Right Reasons?cited by 12
  3. 2024Fairness and Bias Mitigation in Computer Vision: A Surveycited by 12
  4. 2023An Empirical Analysis of Fairness Notions under Differential Privacycited by 9
  5. 2023Unmasking Bias in AI: A Systematic Review of Bias Detection and Mitigation Strategies in Electronic Health Record-based Modelscited by 9
  6. 2025FairPFN: A Tabular Foundation Model for Causal Fairnesscited by 9
  7. 2022Bias-inducing geometries: an exactly solvable data model with fairness implicationscited by 8
  8. 2024FairPFN: Transformers Can do Counterfactual Fairnesscited by 8
  9. 2024What Constitutes a Less Discriminatory Algorithm?cited by 8
  10. 2023Is Your Model "MADD"? A Novel Metric to Evaluate Algorithmic Fairness for Predictive Student Modelscited by 7
  11. 2024Properties of fairness measures in the context of varying class imbalance and protected group ratioscited by 7
  12. 2022Towards Procedural Fairness: Uncovering Biases in How a Toxic Language Classifier Uses Sentiment Informationcited by 6
  13. 2024Improving LLM Group Fairness on Tabular Data via In-Context Learningcited by 6
  14. 2022Error Parity Fairness: Testing for Group Fairness in Regression Taskscited by 5
  15. 2022Algorithmic decision making methods for fair credit scoringcited by 5
  16. 2022BIASeD: Bringing Irrationality into Automated System Designcited by 5
  17. 2023Fairness Hacking: The Malicious Practice of Shrouding Unfairness in Algorithmscited by 5
  18. 2024What is Fair? Defining Fairness in Machine Learning for Healthcited by 5
  19. 2024EARN Fairness: Explaining, Asking, Reviewing, and Negotiating Artificial Intelligence Fairness Metrics Among Stakeholderscited by 5
  20. 2025Investigating Label Bias and Representational Sources of Age-Related Disparities in Medical Segmentationcited by 5
  21. 2023Equalised Odds is not Equal Individual Odds: Post-processing for Group and Individual Fairnesscited by 4
  22. 2023Group-blind optimal transport to group parity and its constrained variantscited by 4
  23. 2023f-FERM: A Scalable Framework for Robust Fair Empirical Risk Minimizationcited by 4
  24. 2024FairICP: Encouraging Equalized Odds via Inverse Conditional Permutationcited by 4
  25. 2024Responsible AI for Earth Observationcited by 4
  26. 2024On Biases in a UK Biobank-based Retinal Image Classification Modelcited by 4
  27. 2025Do Existing Testing Tools Really Uncover Gender Bias in Text-to-Image Models?cited by 4
  28. 2022A survey of Identification and mitigation of Machine Learning algorithmic biases in Image Analysiscited by 3
  29. 2023Uncovering Bias in Personal Informaticscited by 3
  30. 2023FairRec: Fairness Testing for Deep Recommender Systemscited by 3
  31. 2023Equal Confusion Fairness: Measuring Group-Based Disparities in Automated Decision Systemscited by 3
  32. 2024On Explaining Unfairness: An Overviewcited by 3
  33. 2024A Unified View of Group Fairness Tradeoffs Using Partial Information Decompositioncited by 3
  34. 2024A Review of Fairness and A Practical Guide to Selecting Context-Appropriate Fairness Metrics in Machine Learningcited by 3
  35. 2022How Robust is your Fair Model? Exploring the Robustness of Diverse Fairness Strategiescited by 2
  36. 2023Group Fairness with Uncertainty in Sensitive Attributescited by 2
  37. 2023Fair Patient Model: Mitigating Bias in the Patient Representation Learned from the Electronic Health Recordscited by 2
  38. 2023Insights From Insurance for Fair Machine Learningcited by 2

50 most cited of 98 All 98 (JSON)