A survey on bias in machine learning research
Not on the map. Why?
Abstract (arXiv)
Current research on bias in machine learning often focuses on fairness, while overlooking the roots or causes of bias. However, bias was originally defined as a "systematic error," often caused by humans at different stages of the research process. This article aims to bridge the gap between past literature on bias in research by providing taxonomy for potential sources of bias and errors in data and models. The paper focus on bias in machine learning pipelines. Survey analyses over forty potential sources of bias in the machine learning (ML) pipeline, providing clear examples for each. By understanding the sources and consequences of bias in machine learning, better methods can be developed for its detecting and mitigating, leading to fairer, more transparent, and more accurate ML models.
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 66 of its 264 references that are in the map. First posted on arXiv 22 Aug 2023.
Builds on
66 of 264 references.
References · 66
- 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,720
- 2019A Survey on Bias and Fairness in Machine Learningcited by 1,917
- 2018Datasheets for Datasetscited by 1,386
- 2018On the Spectral Bias of Neural Networkscited by 1,341
- 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,312
- 2019Attention is not Explanationcited by 929
- 2023Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedbackcited by 759
- 2014Certifying and removing disparate impactcited by 715
- 2020CrowS-Pairs: A Challenge Dataset for Measuring Social Biases in Masked Language Modelscited by 710
- 2018Mitigating Unwanted Biases with Adversarial Learningcited by 663
- 2019Attention is not not Explanationcited by 636
- 2020Captum: A unified and generic model interpretability library for PyTorchcited by 547
38 more
- 2017Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraintscited by 420
- 2017Algorithmic decision making and the cost of fairnesscited by 403
- 2020Fairness in Machine Learning: A Surveycited by 370
- 2019Unmasking Clever Hans Predictors and Assessing What Machines Really Learncited by 365
- 2018Fairness Without Demographics in Repeated Loss Minimizationcited by 364
- 2019Mitigating Gender Bias in Natural Language Processing: Literature Reviewcited by 287
- 2017WebVision Database: Visual Learning and Understanding from Web Datacited by 262
- 2020Identifying Mislabeled Data using the Area Under the Margin Rankingcited by 261
- 2019Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigationcited by 252
- 2020Large image datasets: A pyrrhic win for computer vision?cited by 237
- 2018Examining Gender and Race Bias in Two Hundred Sentiment Analysis Systemscited by 234
- 2019Informed Machine Learning -- A Taxonomy and Survey of Integrating Knowledge into Learning Systemscited by 205
- 2019A Framework for Understanding Sources of Harm throughout the Machine Learning Life Cyclecited by 193
- 2015Censoring Representations with an Adversarycited by 192
- 2020Adversarial Filters of Dataset Biasescited by 160
- 2020Minimax Pareto Fairness: A Multi Objective Perspectivecited by 153
- 2018The Frontiers of Fairness in Machine Learningcited by 141
- 2019Predictive Biases in Natural Language Processing Models: A Conceptual Framework and Overviewcited by 137
- 2018Troubling Trends in Machine Learning Scholarshipcited by 108
- 2021A Clarification of the Nuances in the Fairness Metrics Landscapecited by 98
- 2018Self-Erasing Network for Integral Object Attentioncited by 95
- 2020Feedback Loop and Bias Amplification in Recommender Systemscited by 90
- 2019Managing Popularity Bias in Recommender Systems with Personalized Re-rankingcited by 82
- 2016A Confidence-Based Approach for Balancing Fairness and Accuracycited by 73
- 2019The Unfairness of Popularity Bias in Recommendationcited by 70
- 2020Towards causal benchmarking of bias in face analysis algorithmscited by 64
- 2021On the Frequency Bias of Generative Modelscited by 60
- 2022Gender Bias in Masked Language Models for Multiple Languagescited by 60
- 2021Controllable Guarantees for Fair Outcomes via Contrastive Information Estimationcited by 45
- 2016A statistical framework for fair predictive algorithmscited by 38
- 2020TACO: Trash Annotations in Context for Litter Detectioncited by 38
- 2020Spatial Frequency Bias in Convolutional Generative Adversarial Networkscited by 35
- 2021TA2N: Two-Stage Action Alignment Network for Few-shot Action Recognitioncited by 35
- 2018Attention-Guided Curriculum Learning for Weakly Supervised Classification and Localization of Thoracic Diseases on Chest Radiographscited by 32
- 2019(De)Constructing Bias on Skin Lesion Datasetscited by 30
- 2021Annotation Inconsistency and Entity Bias in MultiWOZcited by 30
- 2018A review of possible effects of cognitive biases on the interpretation of rule-based machine learning modelscited by 22
- 2017How algorithmic popularity bias hinders or promotes qualitycited by 20
50 most cited of 66 All 66 (JSON)
Built on by
Citing papers per year: 2023: 1; 2024: 1.
2 papers.