# Stereotyping and Bias in the Flickr30K Dataset

arXiv: https://arxiv.org/abs/1605.06083 · HTML: https://mlsanity.com/paper/1605.06083/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 May 2016.

## Built on by (most cited first, 25 of 25)

- [Understanding and Evaluating Racial Biases in Image Captioning](https://mlsanity.com/paper/2106.08503/) (2021)
- [Uncurated Image-Text Datasets: Shedding Light on Demographic Bias](https://mlsanity.com/paper/2304.02828/) (2023)
- [PASS: An ImageNet replacement for self-supervised pretraining without humans](https://mlsanity.com/paper/2109.13228/) (2021)
- [A Decade's Battle on Dataset Bias: Are We There Yet?](https://mlsanity.com/paper/2403.08632/) (2024)
- [Gender Artifacts in Visual Datasets](https://mlsanity.com/paper/2206.09191/) (2022)
- [Measuring Representational Harms in Image Captioning](https://mlsanity.com/paper/2206.07173/) (2022)
- [The World of an Octopus: How Reporting Bias Influences a Language Model's Perception of Color](https://mlsanity.com/paper/2110.08182/) (2021)
- [Underspecification in Scene Description-to-Depiction Tasks](https://mlsanity.com/paper/2210.05815/) (2022)
- [Dimensions of Commonsense Knowledge](https://mlsanity.com/paper/2101.04640/) (2021)
- [Balancing the Picture: Debiasing Vision-Language Datasets with Synthetic Contrast Sets](https://mlsanity.com/paper/2305.15407/) (2023)
- [Survey of Social Bias in Vision-Language Models](https://mlsanity.com/paper/2309.14381/) (2023)
- [A Friendly Face: Do Text-to-Image Systems Rely on Stereotypes when the Input is Under-Specified?](https://mlsanity.com/paper/2302.07159/) (2023)
- [Understanding Bias in Large-Scale Visual Datasets](https://mlsanity.com/paper/2412.01876/) (2024)
- [Computer Vision and Conflicting Values: Describing People with Automated Alt Text](https://mlsanity.com/paper/2105.12754/) (2021)
- [Fairness and Bias Mitigation in Computer Vision: A Survey](https://mlsanity.com/paper/2408.02464/) (2024)
- [Simplicity Bias Leads to Amplified Performance Disparities](https://mlsanity.com/paper/2212.06641/) (2022)
- [Evaluating Bias and Fairness in Gender-Neutral Pretrained Vision-and-Language Models](https://mlsanity.com/paper/2310.17530/) (2023)
- [Fairness in AI Systems: Mitigating gender bias from language-vision models](https://mlsanity.com/paper/2305.01888/) (2023)
- [Quantifying the Gaps Between Translation and Native Perception in Training for Multimodal, Multilingual Retrieval](https://mlsanity.com/paper/2410.02027/) (2024)
- [Measuring Machine Learning Harms from Stereotypes Requires Understanding Who Is Harmed by Which Errors in What Ways](https://mlsanity.com/paper/2402.04420/) (2024)
- [Impressions: Understanding Visual Semiotics and Aesthetic Impact](https://mlsanity.com/paper/2310.17887/) (2023)
- [Attention IoU: Examining Biases in CelebA using Attention Maps](https://mlsanity.com/paper/2503.19846/) (2025)
- [Understanding Gender Bias in AI-Generated Product Descriptions](https://mlsanity.com/paper/2506.05390/) (2025)
- [Common-Sense Bias Modeling for Classification Tasks](https://mlsanity.com/paper/2401.13213/) (2024)
- [Personalized Generative Models for Contextual Debiasing](https://mlsanity.com/paper/2605.26353/) (2026)

## Abstract (arXiv)

An untested assumption behind the crowdsourced descriptions of the images in the Flickr30K dataset (Young et al., 2014) is that they "focus only on the information that can be obtained from the image alone" (Hodosh et al., 2013, p. 859). This paper presents some evidence against this assumption, and provides a list of biases and unwarranted inferences that can be found in the Flickr30K dataset. Finally, it considers methods to find examples of these, and discusses how we should deal with stereotype-driven descriptions in future applications.