mlsanityData to 14 Oct 2026

Bias in Machine Learning Software: Why? How? What to do?

arXiv 2105.12195 ↗First posted 25 May 2021cs.LG cs.SEcited by 93

Not on the map. Why?

Abstract (arXiv)

Increasingly, software is making autonomous decisions in case of criminal sentencing, approving credit cards, hiring employees, and so on. Some of these decisions show bias and adversely affect certain social groups (e.g. those defined by sex, race, age, marital status). Many prior works on bias mitigation take the following form: change the data or learners in multiple ways, then see if any of that improves fairness. Perhaps a better approach is to postulate root causes of bias and then applying some resolution strategy. This paper postulates that the root causes of bias are the prior decisions that affect- (a) what data was selected and (b) the labels assigned to those examples. Our Fair-SMOTE algorithm removes biased labels; and rebalances internal distributions such that based on sensitive attribute, examples are equal in both positive and negative classes. On testing, it was seen that this method was just as effective at reducing bias as prior approaches. Further, models generated via Fair-SMOTE achieve higher performance (measured in terms of recall and F1) than other state-of-the-art fairness improvement algorithms. To the best of our knowledge, measured in terms of number of analyzed learners and datasets, this study is one of the largest studies on bias mitigation yet presented in the literature.

Key facts. Cited by 93 papers in the map (data through 14 Oct 2026). Builds on 15 of its 54 references that are in the map. First posted on arXiv 25 May 2021.

Builds on

15 of 54 references.

References · 15

  1. 2016Equality of Opportunity in Supervised Learningcited by 2,007
  2. 2016Semantics derived automatically from language corpora contain human-like biasescited by 901
  3. 2014Certifying and removing disparate impactcited by 715
  4. 2018Mitigating Unwanted Biases with Adversarial Learningcited by 663
  5. 2018AI Fairness 360: An Extensible Toolkit for Detecting, Understanding, and Mitigating Unwanted Algorithmic Biascited by 319
  6. 2017Fairness in Criminal Justice Risk Assessments: The State of the Artcited by 315
  7. 2019Identifying and Correcting Label Bias in Machine Learningcited by 146
  8. 2017Fairness Testing: Testing Software for Discriminationcited by 126
  9. 2018Automated Directed Fairness Testingcited by 78
  10. 2016A causal framework for discovering and removing direct and indirect discriminationcited by 71
  11. 2020Do the Machine Learning Models on a Crowd Sourced Platform Exhibit Bias? An Empirical Study on Model Fairnesscited by 52
  12. 2018Causal Reasoning for Algorithmic Fairnesscited by 51
3 more
  1. 2020Making Fair ML Software using Trustworthy Explanationcited by 24
  2. 2019Software Engineering for Fairness: A Case Study with Hyperparameter Optimizationcited by 18
  3. 2020Fairkit, Fairkit, on the Wall, Who's the Fairest of Them All? Supporting Data Scientists in Training Fair Modelscited by 9

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 8; 2022: 16; 2023: 27; 2024: 20; 2025: 17; 2026: 5.

93 papers.

Sort:

By area:

  1. 2022Bias Mitigation for Machine Learning Classifiers: A Comprehensive Surveycited by 125
  2. 2023BiasAsker: Measuring the Bias in Conversational AI Systemcited by 53
  3. 2022A Comprehensive Empirical Study of Bias Mitigation Methods for Machine Learning Classifierscited by 30
  4. 2023Latent Imitator: Generating Natural Individual Discriminatory Instances for Black-Box Fairness Testingcited by 23
  5. 2022Fairify: Fairness Verification of Neural Networkscited by 22
  6. 2021FairMask: Better Fairness via Model-based Rebalancing of Protected Attributescited by 19
  7. 2022Towards Understanding Fairness and its Composition in Ensemble Machine Learningcited by 17
  8. 2024New Job, New Gender? Measuring the Social Bias in Image Generation Modelscited by 16
  9. 2021Fair Enough: Searching for Sufficient Measures of Fairnesscited by 14
  10. 2023Towards Fair Machine Learning Software: Understanding and Addressing Model Bias Through Counterfactual Thinkingcited by 14
  11. 2022FairIF: Boosting Fairness in Deep Learning via Influence Functions with Validation Set Sensitive Attributescited by 12
  12. 2023Preventing Discriminatory Decision-making in Evolving Data Streamscited by 12
38 more
  1. 2024OxonFair: A Flexible Toolkit for Algorithmic Fairnesscited by 12
  2. 2024NeuFair: Neural Network Fairness Repair with Dropoutcited by 11
  3. 2022Towards A Holistic View of Bias in Machine Learning: Bridging Algorithmic Fairness and Imbalanced Learningcited by 10
  4. 2023Information-Theoretic Testing and Debugging of Fairness Defects in Deep Neural Networkscited by 10
  5. 2023Fix Fairness, Don't Ruin Accuracy: Performance Aware Fairness Repair using AutoMLcited by 9
  6. 2022Tiny, always-on and fragile: Bias propagation through design choices in on-device machine learning workflowscited by 8
  7. 2024Generative AI voting: fair collective choice is resilient to LLM biases and inconsistenciescited by 8
  8. 2024A Nested Model for AI Design and Validationcited by 8
  9. 2022Understanding Unfairness in Fraud Detection through Model and Data Bias Interactionscited by 7
  10. 2023Bias Behind the Wheel: Fairness Testing of Autonomous Driving Systemscited by 7
  11. 2023On Leakage in Machine Learning Pipelinescited by 7
  12. 2023Causality-Aided Trade-off Analysis for Machine Learning Fairnesscited by 6
  13. 2024A Large-Scale Empirical Study on Improving the Fairness of Image Classification Modelscited by 6
  14. 2024Rethinking Software Engineering in the Foundation Model Era: A Curated Catalogue of Challenges in the Development of Trustworthy FMwarecited by 6
  15. 2025FairMedQA: Benchmarking Bias in Large Language Models for Medical Question Answeringcited by 6
  16. 2021FairBalance: How to Achieve Equalized Odds With Data Pre-processingcited by 5
  17. 2023Fair Oversampling Technique using Heterogeneous Clusterscited by 5
  18. 2023FairWASP: Fast and Optimal Fair Wasserstein Pre-processingcited by 5
  19. 2024Synthetic Tabular Data Generation for Class Imbalance and Fairness: A Comparative Studycited by 5
  20. 2021Fair-SSL: Building fair ML Software with less datacited by 4
  21. 2022Towards Fairness-Aware Multi-Objective Optimizationcited by 4
  22. 2023Fairness-Aware Data Valuation for Supervised Learningcited by 4
  23. 2024Fairness Concerns in App Reviews: A Study on AI-based Mobile Appscited by 4
  24. 2024A Catalog of Fairness-Aware Practices in Machine Learning Engineeringcited by 4
  25. 2021Developing a novel fair-loan-predictor through a multi-sensitive debiasing pipeline: DualFaircited by 3
  26. 2023What are the Machine Learning best practices reported by practitioners on Stack Exchange?cited by 3
  27. 2023RAI4IoE: Responsible AI for Enabling the Internet of Energycited by 3
  28. 2024Diversity Drives Fairness: Ensemble of Higher Order Mutants for Intersectional Fairness of Machine Learning Softwarecited by 3
  29. 2025FairSense: Long-Term Fairness Analysis of ML-Enabled Systemscited by 3
  30. 2022ArabGend: Gender Analysis and Inference on Arabic Twittercited by 2
  31. 2022Enhanced Fairness Testing via Generating Effective Initial Individual Discriminatory Instancescited by 2
  32. 2023Don't Lie to Me: Avoiding Malicious Explanations with STEALTHcited by 2
  33. 2023FITNESS: A Causal De-correlation Approach for Mitigating Bias in Machine Learning Softwarecited by 2
  34. 2025Fairness Testing through Extreme Value Theorycited by 2
  35. 2025Human in the Latent Loop (HILL): Interactively Guiding Model Training Through Human Intuitioncited by 2
  36. 2025SMOTE-DP: Improving Privacy-Utility Tradeoff with Synthetic Datacited by 2
  37. 2025TABFAIRGDT: A Fast Fair Tabular Data Generator using Autoregressive Decision Treescited by 2
  38. 2023A Three-Way Knot: Privacy, Fairness, and Predictive Performance Dynamicscited by 1

50 most cited of 93 All 93 (JSON)