mlsanityData to 7 Oct 2026

Emergent Unfairness in Algorithmic Fairness-Accuracy Trade-Off Research

arXiv 2102.01203 ↗First posted 1 Feb 2021cs.CY cs.AI cs.LGcited by 34

Not on the map. Why?

Abstract (arXiv)

Across machine learning (ML) sub-disciplines, researchers make explicit mathematical assumptions in order to facilitate proof-writing. We note that, specifically in the area of fairness-accuracy trade-off optimization scholarship, similar attention is not paid to the normative assumptions that ground this approach. Such assumptions presume that 1) accuracy and fairness are in inherent opposition to one another, 2) strict notions of mathematical equality can adequately model fairness, 3) it is possible to measure the accuracy and fairness of decisions independent from historical context, and 4) collecting more data on marginalized individuals is a reasonable solution to mitigate the effects of the trade-off. We argue that such assumptions, which are often left implicit and unexamined, lead to inconsistent conclusions: While the intended goal of this work may be to improve the fairness of machine learning models, these unexamined, implicit assumptions can in fact result in emergent unfairness. We conclude by suggesting a concrete path forward toward a potential resolution.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). Builds on 14 of its 76 references that are in the map. First posted on arXiv 1 Feb 2021.

Builds on

14 of 76 references.

References · 14

  1. 2020Scaling Laws for Neural Language Modelscited by 6,411
  2. 2016Equality of Opportunity in Supervised Learningcited by 2,000
  3. 2016Fair prediction with disparate impact: A study of bias in recidivism prediction instrumentscited by 671
  4. 2016Inherent Trade-Offs in the Fair Determination of Risk Scorescited by 582
  5. 2017Algorithmic decision making and the cost of fairnesscited by 402
  6. 2017Fairness in Machine Learning: Lessons from Political Philosophycited by 148
  7. 2019Inherent Tradeoffs in Learning Fair Representationscited by 144
  8. 2018Why Is My Classifier Discriminatory?cited by 143
  9. 2019Problem Formulation and Fairnesscited by 85
  10. 2019Mathematical Notions vs. Human Perception of Fairness: A Descriptive Approach to Fairness for Machine Learningcited by 57
  11. 2019Roles for Computing in Social Changecited by 49
  12. 2018Fairness Behind a Veil of Ignorance: A Welfare Analysis for Automated Decision Makingcited by 46
2 more
  1. 2018Active Fairness in Algorithmic Decision Makingcited by 27
  2. 2016On the (im)possibility of fairnesscited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 5; 2023: 12; 2024: 5; 2025: 9; 2026: 1.

34 papers.

Sort:

  1. 2023Bias Testing and Mitigation in LLM-based Code Generationcited by 53
  2. 2022Accountability in an Algorithmic Society: Relationality, Responsibility, and Robustness in Machine Learningcited by 47
  3. 2023Queer In AI: A Case Study in Community-Led Participatory AIcited by 33
  4. 2023Arbitrariness and Social Prediction: The Confounding Role of Variance in Fair Classificationcited by 31
  5. 2023Factoring the Matrix of Domination: A Critical Review and Reimagination of Intersectionality in AI Fairnesscited by 26
  6. 2021Hyperparameter Optimization Is Deceiving Us, and How to Stop Itcited by 20
  7. 2023Ethical Considerations for Responsible Data Curationcited by 17
  8. 2023Can Fairness be Automated? Guidelines and Opportunities for Fairness-aware AutoMLcited by 16
  9. 2022Don't Throw it Away! The Utility of Unlabeled Data in Fair Decision Makingcited by 12
  10. 2025U-Fair: Uncertainty-based Multimodal Multitask Learning for Fairer Depression Detectioncited by 12
  11. 2024How to be fair? A study of label and selection biascited by 11
  12. 2025AI Mismatches: Identifying Potential Algorithmic Harms Before AI Developmentcited by 10
22 more
  1. 2022Survey on Fairness Notions and Related Tensionscited by 9
  2. 2021Gradual (In)Compatibility of Fairness Criteriacited by 8
  3. 2024What Constitutes a Less Discriminatory Algorithm?cited by 8
  4. 2023Optimization's Neglected Normative Commitmentscited by 4
  5. 2024A Catalog of Fairness-Aware Practices in Machine Learning Engineeringcited by 4
  6. 2022A Justice-Based Framework for the Analysis of Algorithmic Fairness-Utility Trade-Offscited by 3
  7. 2022Distributive Justice as the Foundational Premise of Fair ML: Unification, Extension, and Interpretation of Group Fairness Metricscited by 3
  8. 2024A Human-in-the-Loop Fairness-Aware Model Selection Framework for Complex Fairness Objective Landscapescited by 3
  9. 2023Fair Enough? A map of the current limitations of the requirements to have fair algorithmscited by 2
  10. 2025Causality Is Key to Understand and Balance Multiple Goals in Trustworthy ML and Foundation Modelscited by 2
  11. 2025Aggregating Concepts of Fairness and Accuracy in Prediction Algorithmscited by 2
  12. 2025The Statistical Fairness-Accuracy Frontiercited by 2
  13. 2023A multidomain relational framework to guide institutional AI research and adoptioncited by 1
  14. 2023A Counterfactual Fair Model for Longitudinal Electronic Health Records via Deconfoundercited by 1
  15. 2025Biased Heritage: How Datasets Shape Models in Facial Expression Recognitioncited by 1
  16. 2023From Utilitarian to Rawlsian Designs for Algorithmic Fairnesscited by 0
  17. 2023The Flawed Foundations of Fair Machine Learningcited by 0
  18. 2024Towards Fairness in Provably Communication-Efficient Federated Recommender Systemscited by 0
  19. 2025You Only Debias Once: Towards Flexible Accuracy-Fairness Trade-offs at Inference Timecited by 0
  20. 2025On the Correlation between Individual Fairness and Predictive Accuracy in Probabilistic Modelscited by 0
  21. 2025The Effect of Enforcing Fairness on Reshaping Explanations in Machine Learning Modelscited by 0
  22. 2026No evaluation without fair representation : Impact of label and selection bias on the evaluation, performance and mitigation of classification modelscited by 0