mlsanityData to 14 Oct 2026

Uncurated Image-Text Datasets: Shedding Light on Demographic Bias

arXiv 2304.02828 ↗First posted 6 Apr 2023cs.CV cs.CYcited by 52

Not on the map. Why?

Abstract (arXiv)

The increasing tendency to collect large and uncurated datasets to train vision-and-language models has raised concerns about fair representations. It is known that even small but manually annotated datasets, such as MSCOCO, are affected by societal bias. This problem, far from being solved, may be getting worse with data crawled from the Internet without much control. In addition, the lack of tools to analyze societal bias in big collections of images makes addressing the problem extremely challenging. Our first contribution is to annotate part of the Google Conceptual Captions dataset, widely used for training vision-and-language models, with four demographic and two contextual attributes. Our second contribution is to conduct a comprehensive analysis of the annotations, focusing on how different demographic groups are represented. Our last contribution lies in evaluating three prevailing vision-and-language tasks: image captioning, text-image CLIP embeddings, and text-to-image generation, showing that societal bias is a persistent problem in all of them.

Key facts. Cited by 52 papers in the map (data through 14 Oct 2026). Builds on 45 of its 62 references that are in the map. First posted on arXiv 6 Apr 2023.

Builds on

45 of 62 references.

References · 45

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  8. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  9. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  10. 2015VQA: Visual Question Answeringcited by 2,506
  11. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
  12. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
33 more
  1. 2015Microsoft COCO Captions: Data Collection and Evaluation Servercited by 1,462
  2. 2015Flickr30k Entities: Collecting Region-to-Phrase Correspondences for Richer Image-to-Sentence Modelscited by 1,362
  3. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,214
  4. 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
  5. 2014Show and Tell: A Neural Image Caption Generatorcited by 879
  6. 2022OFA: Unifying Architectures, Tasks, and Modalities Through a Simple Sequence-to-Sequence Learning Frameworkcited by 745
  7. 2021ClipCap: CLIP Prefix for Image Captioningcited by 559
  8. 2019Unicoder-VL: A Universal Encoder for Vision and Language by Cross-modal Pre-trainingcited by 451
  9. 2017Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraintscited by 420
  10. 2021Multimodal datasets: misogyny, pornography, and malignant stereotypescited by 327
  11. 2021Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotationscited by 314
  12. 2022Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scalecited by 262
  13. 2020Large image datasets: A pyrrhic win for computer vision?cited by 237
  14. 2018Women also Snowboard: Overcoming Bias in Captioning Modelscited by 236
  15. 2020ERNIE-ViL: Knowledge Enhanced Vision-Language Representations Through Scene Graphcited by 211
  16. 2021RedCaps: web-curated image-text data created by the people, for the peoplecited by 180
  17. 2019Towards Fairer Datasets: Filtering and Balancing the Distribution of the People Subtree in the ImageNet Hierarchycited by 156
  18. 2020REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasetscited by 136
  19. 2019Towards a Critical Race Methodology in Algorithmic Fairnesscited by 125
  20. 2021Understanding and Evaluating Racial Biases in Image Captioningcited by 108
  21. 2022A Prompt Array Keeps the Bias Away: Debiasing Vision-Language Models with Adversarial Learningcited by 104
  22. 2021Are Gender-Neutral Queries Really Gender-Neutral? Mitigating Gender Bias in Image Searchcited by 90
  23. 2019Predictive Inequity in Object Detectioncited by 77
  24. 2020Measuring Social Biases in Grounded Vision and Language Embeddingscited by 61
  25. 2021Directional Bias Amplificationcited by 54
  26. 2021Worst of Both Worlds: Biases Compound in Pre-trained Vision-and-Language Modelscited by 52
  27. 2020Mitigating Gender Bias in Captioning Systemscited by 50
  28. 2022Gender and Racial Bias in Visual Question Answering Datasetscited by 39
  29. 2022Gender Artifacts in Visual Datasetscited by 38
  30. 2022Quantifying Societal Bias Amplification in Image Captioningcited by 37
  31. 2019Exposing and Correcting the Gender Bias in Image Captioning Datasets and Modelscited by 36
  32. 2022Measuring Representational Harms in Image Captioningcited by 36
  33. 2016Stereotyping and Bias in the Flickr30K Datasetcited by 25

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 7; 2024: 22; 2025: 16; 2026: 7.

52 papers.

Sort:

By area:

  1. 2024No "Zero-Shot" Without Exponential Data: Pretraining Concept Frequency Determines Multimodal Model Performancecited by 75
  2. 2023The Bias Amplification Paradox in Text-to-Image Generationcited by 70
  3. 2024Survey of Bias In Text-to-Image Generation: Definition, Evaluation, and Mitigationcited by 66
  4. 2024PromptCharm: Text-to-Image Generation through Multi-modal Prompting and Refinementcited by 62
  5. 2024Design Principles for Generative AI Applicationscited by 57
  6. 2023SocialCounterfactuals: Probing and Mitigating Intersectional Social Biases in Vision-Language Models with Counterfactual Examplescited by 53
  7. 2023Inspecting the Geographical Representativeness of Images from Text-to-Image Modelscited by 44
  8. 2024ViSAGe: A Global-Scale Analysis of Visual Stereotypes in Text-to-Image Generationcited by 27
  9. 2024Securing Large Language Models: Addressing Bias, Misinformation, and Prompt Attackscited by 26
  10. 2023Stable Diffusion Exposed: Gender Bias from Prompt to Imagecited by 25
  11. 2024Would Deep Generative Models Amplify Bias in Future Models?cited by 24
  12. 2024Do Vision & Language Decoders use Images and Text equally? How Self-consistent are their Explanations?cited by 24
38 more
  1. 2024SCoFT: Self-Contrastive Fine-Tuning for Equitable Image Generationcited by 19
  2. 2024SANER: Annotation-free Societal Attribute Neutralizer for Debiasing CLIPcited by 19
  3. 2025Scaling Pre-training to One Hundred Billion Data for Vision Language Modelscited by 19
  4. 2024Joint Vision-Language Social Bias Removal for CLIPcited by 18
  5. 2023Model-Agnostic Gender Debiased Image Captioningcited by 17
  6. 2023Quantifying Bias in Text-to-Image Generative Modelscited by 16
  7. 2024No Filter: Cultural and Socioeconomic Diversity in Contrastive Vision-Language Modelscited by 16
  8. 2024Revealing and Reducing Gender Biases in Vision and Language Assistants (VLAs)cited by 15
  9. 2024Fairness and Bias Mitigation in Computer Vision: A Surveycited by 13
  10. 2024FairDeDup: Detecting and Mitigating Vision-Language Fairness Disparities in Semantic Dataset Deduplicationcited by 11
  11. 2024Value Imprint: A Technique for Auditing the Human Values Embedded in RLHF Datasetscited by 11
  12. 2023Adventures of Trustworthy Vision-Language Models: A Surveycited by 10
  13. 2025Exploring Bias in over 100 Text-to-Image Generative Modelscited by 9
  14. 2024Resampled Datasets Are Not Enough: Mitigating Societal Bias Beyond Single Attributescited by 7
  15. 2024Image-Perfect Imperfections: Safety, Bias, and Authenticity in the Shadow of Text-To-Image Model Evolutioncited by 7
  16. 2024FairQueue: Rethinking Prompt Learning for Fair Text-to-Image Generationcited by 7
  17. 2025Data Leakage in Visual Datasetscited by 7
  18. 2024Visual Data Diagnosis and Debiasing with Concept Graphscited by 6
  19. 2026Interpretable Debiasing of Vision-Language Models for Social Fairnesscited by 6
  20. 2024Annotations on a Budget: Leveraging Geo-Data Similarity to Balance Model Performance and Annotation Costcited by 5
  21. 2025Bias in Gender Bias Benchmarks: How Spurious Features Distort Evaluationcited by 5
  22. 2025Person-Centric Annotations of LAION-400M: Auditing Bias and Its Transfer to Modelscited by 4
  23. 2025How I Met Your Bias: Investigating Bias Amplification in Diffusion Modelscited by 4
  24. 2025Understanding Gender Bias in AI-Generated Product Descriptionscited by 3
  25. 2025Sparse Autoencoders are Topic Modelscited by 3
  26. 2024A Taxonomy of Challenges to Curating Fair Datasetscited by 2
  27. 2025Cultural Awareness in Vision-Language Models: A Cross-Country Explorationcited by 2
  28. 2025LOTUS: A Leaderboard for Detailed Image Captioning from Quality to Societal Bias and User Preferencescited by 2
  29. 2025From Global to Local: Social Bias Transfer in CLIPcited by 2
  30. 2025The Iconicity of the Generated Imagecited by 2
  31. 2026Measuring Social Bias in Vision-Language Models with Face-Only Counterfactuals from Real Photoscited by 2
  32. 2025Position: Restructuring of Categories and Implementation of Guidelines Essential for VLM Adoption in Healthcarecited by 1
  33. 2025MineTheGap: Automatic Mining of Biases in Text-to-Image Modelscited by 1
  34. 2026Gender Artifacts from Art History to Text-to-Image Generationcited by 1
  35. 2026Investigating Social Bias in Narrative Image Generationcited by 1
  36. 2025TEDI: Trustworthy and Ethical Dataset Indicators to Analyze and Compare Dataset Documentationcited by 0
  37. 2025"Draw me a curator" Examining the visual stereotyping of a cultural services profession by generative AIcited by 0
  38. 2026BlendFusion -- Scalable Synthetic Data Generation for Diffusion Model Trainingcited by 0

50 most cited of 52 All 52 (JSON)