# Gender Artifacts in Visual Datasets

arXiv: https://arxiv.org/abs/2206.09191 · HTML: https://mlsanity.com/paper/2206.09191/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Builds on 50 of its 87 references that are in the map.
- First posted on arXiv 18 Jun 2022.

## Builds on (most cited first, 30 of 50)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Equality of Opportunity in Supervised Learning](https://mlsanity.com/paper/1610.02413/) (2016)
- [Datasheets for Datasets](https://mlsanity.com/paper/1803.09010/) (2018)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [Language (Technology) is Power: A Critical Survey of "Bias" in NLP](https://mlsanity.com/paper/2005.14050/) (2020)
- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [Semantics derived automatically from language corpora contain human-like biases](https://mlsanity.com/paper/1608.07187/) (2016)
- [Image Segmentation Using Deep Learning: A Survey](https://mlsanity.com/paper/2001.05566/) (2020)
- [The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale](https://mlsanity.com/paper/1811.00982/) (2018)
- [Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraints](https://mlsanity.com/paper/1707.09457/) (2017)
- [All Models are Wrong, but Many are Useful: Learning a Variable's Importance by Studying an Entire Class of Prediction Models Simultaneously](https://mlsanity.com/paper/1801.01489/) (2018)
- [Fairness in Machine Learning: A Survey](https://mlsanity.com/paper/2010.04053/) (2020)
- [Noise or Signal: The Role of Image Backgrounds in Object Recognition](https://mlsanity.com/paper/2006.09994/) (2020)
- [Data and its (dis)contents: A survey of dataset development and use in machine learning research](https://mlsanity.com/paper/2012.05345/) (2020)
- [Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigation](https://mlsanity.com/paper/1911.11834/) (2019)
- [Large image datasets: A pyrrhic win for computer vision?](https://mlsanity.com/paper/2006.16923/) (2020)
- [Women also Snowboard: Overcoming Bias in Captioning Models](https://mlsanity.com/paper/1803.09797/) (2018)
- [A Comprehensive Survey of Deep Learning for Image Captioning](https://mlsanity.com/paper/1810.04020/) (2018)
- [Towards Fairer Datasets: Filtering and Balancing the Distribution of the People Subtree in the ImageNet Hierarchy](https://mlsanity.com/paper/1912.07726/) (2019)
- [Reading Race: AI Recognises Patient's Racial Identity In Medical Images](https://mlsanity.com/paper/2107.10356/) (2021)
- [No Classification without Representation: Assessing Geodiversity Issues in Open Data Sets for the Developing World](https://mlsanity.com/paper/1711.08536/) (2017)
- [Does Object Recognition Work for Everyone?](https://mlsanity.com/paper/1906.02659/) (2019)
- [Towards Accountability for Machine Learning Datasets: Practices from Software Engineering and Infrastructure](https://mlsanity.com/paper/2010.13561/) (2020)
- [REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasets](https://mlsanity.com/paper/2004.07999/) (2020)
- [Fair Attribute Classification through Latent Space De-biasing](https://mlsanity.com/paper/2012.01469/) (2020)
- [Lessons from Archives: Strategies for Collecting Sociocultural Data in Machine Learning](https://mlsanity.com/paper/1912.10389/) (2019)

## Built on by (most cited first, 30 of 38)

- [On Measuring Faithfulness or Self-consistency of Natural Language Explanations](https://mlsanity.com/paper/2311.07466/) (2023)
- [FACET: Fairness in Computer Vision Evaluation Benchmark](https://mlsanity.com/paper/2309.00035/) (2023)
- [Uncurated Image-Text Datasets: Shedding Light on Demographic Bias](https://mlsanity.com/paper/2304.02828/) (2023)
- [VisoGender: A dataset for benchmarking gender bias in image-text pronoun resolution](https://mlsanity.com/paper/2306.12424/) (2023)
- [TIBET: Identifying and Evaluating Biases in Text-to-Image Generative Models](https://mlsanity.com/paper/2312.01261/) (2023)
- [MM-SHAP: A Performance-agnostic Metric for Measuring Multimodal Contributions in Vision and Language Models & Tasks](https://mlsanity.com/paper/2212.08158/) (2022)
- [Bias Mimicking: A Simple Sampling Approach for Bias Mitigation](https://mlsanity.com/paper/2209.15605/) (2022)
- [StoryBench: A Multifaceted Benchmark for Continuous Story Visualization](https://mlsanity.com/paper/2308.11606/) (2023)
- [Balancing the Picture: Debiasing Vision-Language Datasets with Synthetic Contrast Sets](https://mlsanity.com/paper/2305.15407/) (2023)
- [Measuring Progress in Fine-grained Vision-and-Language Understanding](https://mlsanity.com/paper/2305.07558/) (2023)
- [Consensus and Subjectivity of Skin Tone Annotation for ML Fairness](https://mlsanity.com/paper/2305.09073/) (2023)
- [Men Also Do Laundry: Multi-Attribute Bias Amplification](https://mlsanity.com/paper/2210.11924/) (2022)
- [Harm Amplification in Text-to-Image Models](https://mlsanity.com/paper/2402.01787/) (2024)
- [Understanding Bias in Large-Scale Visual Datasets](https://mlsanity.com/paper/2412.01876/) (2024)
- [Fairness and Bias Mitigation in Computer Vision: A Survey](https://mlsanity.com/paper/2408.02464/) (2024)
- [From Fake to Real: Pretraining on Balanced Synthetic Images to Prevent Spurious Correlations in Image Recognition](https://mlsanity.com/paper/2308.04553/) (2023)
- [Social Perception of Faces in a Vision-Language Model](https://mlsanity.com/paper/2408.14435/) (2024)
- [A Multidimensional Analysis of Social Biases in Vision Transformers](https://mlsanity.com/paper/2308.01948/) (2023)
- [Data Leakage in Visual Datasets](https://mlsanity.com/paper/2508.17416/) (2025)
- [Weakly-Supervised Learning of Visual Relations in Multimodal Pretraining](https://mlsanity.com/paper/2305.14281/) (2023)
- [PRISM: Reducing Spurious Implicit Biases in Vision-Language Models with LLM-Guided Embedding Projection](https://mlsanity.com/paper/2507.08979/) (2025)
- [Bias in Gender Bias Benchmarks: How Spurious Features Distort Evaluation](https://mlsanity.com/paper/2509.07596/) (2025)
- [Person-Centric Annotations of LAION-400M: Auditing Bias and Its Transfer to Models](https://mlsanity.com/paper/2510.03721/) (2025)
- [Variation of Gender Biases in Visual Recognition Models Before and After Finetuning](https://mlsanity.com/paper/2303.07615/) (2023)
- [What Are You Doing? A Closer Look at Controllable Human Video Generation](https://mlsanity.com/paper/2503.04666/) (2025)
- [Fairness in Autonomous Driving: Towards Understanding Confounding Factors in Object Detection under Challenging Weather](https://mlsanity.com/paper/2406.00219/) (2024)
- [A Taxonomy of Challenges to Curating Fair Datasets](https://mlsanity.com/paper/2406.06407/) (2024)
- [A Systematic Review of Open Datasets Used in Text-to-Image (T2I) Gen AI Model Safety](https://mlsanity.com/paper/2503.00020/) (2025)
- [BiasConnect: Investigating Bias Interactions in Text-to-Image Models](https://mlsanity.com/paper/2503.09763/) (2025)
- [From Global to Local: Social Bias Transfer in CLIP](https://mlsanity.com/paper/2508.17750/) (2025)

## Abstract (arXiv)

Gender biases are known to exist within large-scale visual datasets and can be reflected or even amplified in downstream models. Many prior works have proposed methods for mitigating gender biases, often by attempting to remove gender expression information from images. To understand the feasibility and practicality of these approaches, we investigate what textitgender artifacts exist within large-scale visual datasets. We define a textitgender artifact as a visual cue that is correlated with gender, focusing specifically on those cues that are learnable by a modern image classifier and have an interpretable human corollary. Through our analyses, we find that gender artifacts are ubiquitous in the COCO and OpenImages datasets, occurring everywhere from low-level information (e.g., the mean value of the color channels) to the higher-level composition of the image (e.g., pose and location of people). Given the prevalence of gender artifacts, we claim that attempts to remove gender artifacts from such datasets are largely infeasible. Instead, the responsibility lies with researchers and practitioners to be aware that the distribution of images within datasets is highly gendered and hence develop methods which are robust to these distributional shifts across groups.