mlsanityData to 14 Oct 2026

Applications and Challenges of Fairness APIs in Machine Learning Software

arXiv 2508.16377 ↗First posted 22 Aug 2025cs.LG cs.SEcited by 0

Not on the map. Why?

Abstract (arXiv)

Machine Learning software systems are frequently used in our day-to-day lives. Some of these systems are used in various sensitive environments to make life-changing decisions. Therefore, it is crucial to ensure that these AI/ML systems do not make any discriminatory decisions for any specific groups or populations. In that vein, different bias detection and mitigation open-source software libraries (aka API libraries) are being developed and used. In this paper, we conduct a qualitative study to understand in what scenarios these open-source fairness APIs are used in the wild, how they are used, and what challenges the developers of these APIs face while developing and adopting these libraries. We have analyzed 204 GitHub repositories (from a list of 1885 candidate repositories) which used 13 APIs that are developed to address bias in ML software. We found that these APIs are used for two primary purposes (i.e., learning and solving real-world problems), targeting 17 unique use-cases. Our study suggests that developers are not well-versed in bias detection and mitigation; they face lots of troubleshooting issues, and frequently ask for opinions and resources. Our findings can be instrumental for future bias-related software engineering research, and for guiding educators in developing more state-of-the-art curricula.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 36 of its 115 references that are in the map. First posted on arXiv 22 Aug 2025.

Builds on

36 of 115 references.

References · 36

  1. 2016Equality of Opportunity in Supervised Learningcited by 2,007
  2. 2019A Survey on Bias and Fairness in Machine Learningcited by 1,917
  3. 2020Language (Technology) is Power: A Critical Survey of "Bias" in NLPcited by 956
  4. 2016Semantics derived automatically from language corpora contain human-like biasescited by 901
  5. 2017Counterfactual Fairnesscited by 804
  6. 2014Certifying and removing disparate impactcited by 715
  7. 2016Fair prediction with disparate impact: A study of bias in recidivism prediction instrumentscited by 672
  8. 2018Mitigating Unwanted Biases with Adversarial Learningcited by 663
  9. 2018A Reductions Approach to Fair Classificationcited by 587
  10. 2017On Fairness and Calibrationcited by 406
  11. 2017Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairnesscited by 384
  12. 2016Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatmentcited by 369
24 more
  1. 2018Fairness Without Demographics in Repeated Loss Minimizationcited by 364
  2. 2018AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Biascited by 319
  3. 2015The Variational Fair Autoencodercited by 223
  4. 2017Data Decisions and Theoretical Implications when Adversarially Learning Fair Representationscited by 165
  5. 2018Classification with Fairness Constraints: A Meta-Algorithm with Provable Guaranteescited by 146
  6. 2020Algorithmic Fairnesscited by 143
  7. 2018An Intersectional Definition of Fairnesscited by 127
  8. 2017Fairness Testing: Testing Software for Discriminationcited by 126
  9. 2018A Unified Approach to Quantifying Algorithmic Unfairness: Measuring Individual & Group Unfairness via Inequality Indicescited by 97
  10. 2017Beyond Parity: Fairness Objectives for Collaborative Filteringcited by 93
  11. 2021Bias in Machine Learning Software: Why? How? What to do?cited by 93
  12. 2017On Formalizing Fairness in Prediction with Machine Learningcited by 65
  13. 2020Do the Machine Learning Models on a Crowd Sourced Platform Exhibit Bias? An Empirical Study on Model Fairnesscited by 52
  14. 2021Fair Preprocessing: Towards Understanding Compositional Fairness of Data Transformers in Machine Learning Pipelinecited by 49
  15. 2021BiasFinder: Metamorphic Test Generation to Uncover Bias for Sentiment Analysis Systemscited by 24
  16. 2021Faults in Deep Reinforcement Learning Programs: A Taxonomy and A Detection Approachcited by 18
  17. 2018Fair lending needs explainable models for responsible recommendationcited by 12
  18. 2021BiasRV: Uncovering Biased Sentiment Predictions at Runtimecited by 12
  19. 2019FAT Forensics: A Python Toolbox for Algorithmic Fairness, Accountability and Transparencycited by 11
  20. 2021The impact of using biased performance metrics on software defect prediction researchcited by 4
  21. 2021Quality Assurance Challenges for Machine Learning Software Applications During Software Development Life Cycle Phasescited by 4
  22. 2021Fair-SSL: Building fair ML Software with less datacited by 4
  23. 2018Expressions of Sentiments During Code Reviews: Male vs. Femalecited by 3
  24. 2021An Empirical Study of Developer Discussions on Low-Code Software Development Challengescited by 2

Built on by

Not cited yet.