mlsanityData to 7 Oct 2026

A Survey on Bias in Visual Datasets

arXiv 2107.07919 ↗First posted 16 Jul 2021cs.CVcited by 84

Not on the map. Why?

Abstract (arXiv)

Computer Vision (CV) has achieved remarkable results, outperforming humans in several tasks. Nonetheless, it may result in significant discrimination if not handled properly as CV systems highly depend on the data they are fed with and can learn and amplify biases within such data. Thus, the problems of understanding and discovering biases are of utmost importance. Yet, there is no comprehensive survey on bias in visual datasets. Hence, this work aims to: i) describe the biases that might manifest in visual datasets; ii) review the literature on methods for bias discovery and quantification in visual datasets; iii) discuss existing attempts to collect bias-aware visual datasets. A key conclusion of our study is that the problem of bias discovery and quantification in visual datasets is still open, and there is room for improvement in terms of both methods and the range of biases that can be addressed. Moreover, there is no such thing as a bias-free dataset, so scientists and practitioners must become aware of the biases in their datasets and make them explicit. To this end, we propose a checklist to spot different types of bias during visual dataset collection.

Key facts. Cited by 84 papers in the map (data through 7 Oct 2026). Builds on 48 of its 114 references that are in the map. First posted on arXiv 16 Jul 2021.

Builds on

48 of 114 references.

References · 48

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  3. 2016Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddingscited by 1,304
  4. 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,282
  5. 2018A Survey Of Methods For Explaining Black Box Modelscited by 970
  6. 2016Semantics derived automatically from language corpora contain human-like biasescited by 894
  7. 2016Fair prediction with disparate impact: A study of bias in recidivism prediction instrumentscited by 671
  8. 2016Inherent Trade-Offs in the Fair Determination of Risk Scorescited by 582
  9. 2013DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognitioncited by 422
  10. 2017Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraintscited by 420
  11. 2019Measurement and Fairnesscited by 299
  12. 2018Learning Not to Learn: Training Deep Neural Networks with Biased Datacited by 255
36 more
  1. 2020The Creation and Detection of Deepfakes: A Surveycited by 246
  2. 2020Large image datasets: A pyrrhic win for computer vision?cited by 237
  3. 2017An introduction to Topological Data Analysis: fundamental and practical aspects for data scientistscited by 171
  4. 2019Towards Fairer Datasets: Filtering and Balancing the Distribution of the People Subtree in the ImageNet Hierarchycited by 156
  5. 2017No Classification without Representation: Assessing Geodiversity Issues in Open Data Sets for the Developing Worldcited by 152
  6. 2019Identifying and Correcting Label Bias in Machine Learningcited by 146
  7. 2020REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasetscited by 136
  8. 2019Towards a Critical Race Methodology in Algorithmic Fairnesscited by 124
  9. 2020Image Representations Learned With Unsupervised Pre-Training Contain Human-like Biasescited by 107
  10. 2013Discrimination in Online Ad Deliverycited by 93
  11. 2020Demographic Bias in Biometrics: A Survey on an Emerging Challengecited by 93
  12. 2020Don't Judge an Object by Its Context: Learning to Overcome Contextual Biascited by 91
  13. 2015A Deeper Look at Dataset Biascited by 86
  14. 2021A Comprehensive Study on Face Recognition Biases Beyond Demographicscited by 86
  15. 2020Face Recognition: Too Bias, or Not Too Bias?cited by 82
  16. 2021Towards Measuring Fairness in AI: the Casual Conversations Datasetcited by 80
  17. 2016Discovering Causal Signals in Imagescited by 79
  18. 2019Diversity in Facescited by 77
  19. 2019Predictive Inequity in Object Detectioncited by 77
  20. 2016Building a Large Scale Dataset for Image Emotion Recognition: The Fine Print and The Benchmarkcited by 71
  21. 2020Towards causal benchmarking of bias in face analysis algorithmscited by 64
  22. 2021The Use and Misuse of Counterfactuals in Ethical Machine Learningcited by 52
  23. 2020Detect and Correct Bias in Multi-Site Neuroimaging Datasetscited by 44
  24. 2020Learning to Model and Ignore Dataset Bias with Mixed Capacity Ensemblescited by 43
  25. 2018Contemplating Visual Emotions: Understanding and Overcoming Dataset Biascited by 41
  26. 2018SCUT-FBP5500: A Diverse Benchmark Dataset for Multi-Paradigm Facial Beauty Predictioncited by 33
  27. 2020MAAD-Face: A Massively Annotated Attribute Dataset for Face Imagescited by 31
  28. 2020Between Subjectivity and Imposition: Power Dynamics in Data Annotation for Computer Visioncited by 25
  29. 2019Auditing ImageNet: Towards a Model-driven Framework for Annotating Demographic Attributes of Large-Scale Image Datasetscited by 24
  30. 2021Problematic Machine Behavior: A Systematic Literature Review of Algorithm Auditscited by 20
  31. 2020Investigating Bias in Deep Face Analysis: The KANFace Dataset and Empirical Studycited by 18
  32. 2019Predicting the Politics of an Image Using Webly Supervised Datacited by 10
  33. 2020The Criminality From Face Illusioncited by 8
  34. 2020Gender Classification and Bias Mitigation in Facial Imagescited by 8
  35. 2016Inferring Gender from Names on the Web: A Comparative Evaluation of Gender Detection Methodscited by 6
  36. 2019The Profiling Potential of Computer Vision and the Challenge of Computational Empiricismcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 11; 2023: 17; 2024: 30; 2025: 20; 2026: 6.

84 papers.

Sort:

By area:

  1. 2023Towards Unbounded Machine Unlearningcited by 261
  2. 2022Towards Adversarial Evaluations for Inexact Machine Unlearningcited by 82
  3. 2023X-InstructBLIP: A Framework for aligning X-Modal instruction-aware representations to LLMs and Emergent Cross-modal Reasoningcited by 78
  4. 2023SocialCounterfactuals: Probing and Mitigating Intersectional Social Biases in Vision-Language Models with Counterfactual Examplescited by 53
  5. 2022Representation Bias in Data: A Survey on Identification and Resolution Techniquescited by 37
  6. 2023Vision-Language Models Performing Zero-Shot Tasks Exhibit Gender-based Disparitiescited by 31
  7. 2024Meerkat: Audio-Visual Large Language Model for Grounding in Space and Timecited by 26
  8. 2024Would Deep Generative Models Amplify Bias in Future Models?cited by 24
  9. 2023FLAC: Fairness-Aware Representation Learning by Suppressing Attribute-Class Associationscited by 18
  10. 2023Multi-dimensional discrimination in Law and Machine Learning -- A comparative overviewcited by 15
  11. 2023Towards Fair Face Verification: An In-depth Analysis of Demographic Biasescited by 14
  12. 2022Large-scale Bilingual Language-Image Contrastive Learningcited by 12
38 more
  1. 2023Metrics for Dataset Demographic Bias: A Case Study on Facial Expression Recognitioncited by 12
  2. 2024T2Vs Meet VLMs: A Scalable Multimodal Dataset for Visual Harmfulness Recognitioncited by 10
  3. 2024Efficient Bias Mitigation Without Privileged Informationcited by 8
  4. 2022Trusted Media Challenge Dataset and User Studycited by 7
  5. 2022Diagnostics for Deep Neural Networks with Automated Copy/Paste Attackscited by 6
  6. 2024Visual Data Diagnosis and Debiasing with Concept Graphscited by 6
  7. 2025Intrinsic Bias is Predicted by Pretraining Data and Correlates with Downstream Performance in Vision-Language Encoderscited by 6
  8. 2022A Survey of Dataset Refinement for Problems in Computer Vision Datasetscited by 5
  9. 2022Casual Conversations v2: Designing a large consent-driven dataset to measure algorithmic bias and robustnesscited by 5
  10. 2023DSAP: Analyzing Bias Through Demographic Comparison of Datasetscited by 5
  11. 2024Limitations of Data-Driven Spectral Reconstruction -- An Optics-Aware Analysiscited by 5
  12. 2024Can the accuracy bias by facial hairstyle be reduced through balancing the training data?cited by 5
  13. 2024Less can be more: representational vs. stereotypical gender bias in facial expression recognitioncited by 5
  14. 2024BAdd: Bias Mitigation through Bias Additioncited by 5
  15. 2024MAVias: Mitigate any Visual Biascited by 5
  16. 2022VIPHY: Probing "Visible" Physical Commonsense Knowledgecited by 4
  17. 2023An investigation of challenges encountered when specifying training data and runtime monitors for safety critical ML applicationscited by 4
  18. 2024Visual Objectification in Films: Towards a New AI Task for Video Interpretationcited by 4
  19. 2024Exploring the Impact of Dataset Bias on Dataset Distillationcited by 4
  20. 2025Decoupling Template Bias in CLIP: Harnessing Empty Prompts for Enhanced Few-Shot Learningcited by 4
  21. 2022A Tour of Visualization Techniques for Computer Vision Datasetscited by 3
  22. 2023Look Beyond Bias with Entropic Adversarial Data Augmentationcited by 3
  23. 2023Attribute Annotation and Bias Evaluation in Visual Datasets for Autonomous Drivingcited by 3
  24. 2024Analyzing and Mitigating Bias for Vulnerable Classes: Towards Balanced Representation in Datasetcited by 3
  25. 2024Debiasing Machine Unlearning with Counterfactual Examplescited by 3
  26. 2025Attention IoU: Examining Biases in CelebA using Attention Mapscited by 3
  27. 2025Enhancements for Developing a Comprehensive AI Fairness Assessment Standardcited by 3
  28. 2026An Illusion of Unlearning? Assessing Machine Unlearning Through Internal Representationscited by 3
  29. 2023Beyond Demographic Parity: Redefining Equal Treatmentcited by 2
  30. 2023RK-core: An Established Methodology for Exploring the Hierarchical Structure within Datasetscited by 2
  31. 2024Runtime Monitoring and Enforcement of Conditional Fairness in Generative AIscited by 2
  32. 2024Machine Unlearning for Medical Imagingcited by 2
  33. 2024Integrating Deep Learning in Cardiology: A Comprehensive Review of Atrial Fibrillation, Left Atrial Scar Segmentation, and the Frontiers of State-of-the-Art Techniquescited by 2
  34. 2024Exploring Fusion Techniques in Multimodal AI-Based Recruitment: Insights from FairCVdbcited by 2
  35. 2024AEye: A Visualization Tool for Image Datasetscited by 2
  36. 2024Statistical Challenges with Dataset Construction: Why You Will Never Have Enough Imagescited by 2
  37. 2024GradBias: Unveiling Word Influence on Bias in Text-to-Image Generative Modelscited by 2
  38. 2024An Open API Architecture to Discover the Trustworthy Explanation of Cloud AI Servicescited by 2

50 most cited of 84 All 84 (JSON)