mlsanityData to 14 Oct 2026

Multimodal datasets: misogyny, pornography, and malignant stereotypes

arXiv 2110.01963 ↗First posted 5 Oct 2021cs.CYcited by 327

Not on the map. Why?

Cited by 53 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We have now entered the era of trillion parameter machine learning models trained on billion-sized datasets scraped from the internet. The rise of these gargantuan datasets has given rise to formidable bodies of critical work that has called for caution while generating these large datasets. These address concerns surrounding the dubious curation practices used to generate these datasets, the sordid quality of alt-text data available on the world wide web, the problematic content of the CommonCrawl dataset often used as a source for training large language models, and the entrenched biases in large-scale visio-linguistic models (such as OpenAI's CLIP model) trained on opaque datasets (WebImageText). In the backdrop of these specific calls of caution, we examine the recently released LAION-400M dataset, which is a CLIP-filtered dataset of Image-Alt-text pairs parsed from the Common-Crawl dataset. We found that the dataset contains, troublesome and explicit images and text pairs of rape, pornography, malign stereotypes, racist and ethnic slurs, and other extremely problematic content. We outline numerous implications, concerns and downstream harms regarding the current state of large scale datasets while raising open questions for various stakeholders including the AI community, regulators, policy makers and data subjects.

Key facts. Cited by 327 papers in the map (data through 14 Oct 2026). First posted on arXiv 5 Oct 2021.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 3; 2022: 65; 2023: 89; 2024: 76; 2025: 62; 2026: 32.

327 papers.

Sort:

By area:

  1. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  2. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  3. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  4. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  5. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
  6. 2022Scaling Autoregressive Models for Content-Rich Text-to-Image Generationcited by 1,295
  7. 2022GPT-NeoX-20B: An Open-Source Autoregressive Language Modelcited by 713
  8. 2021FLAVA: A Foundational Language And Vision Alignment Modelcited by 649
  9. 2023DataComp: In search of the next generation of multimodal datasetscited by 644
  10. 2022Perceiver-Actor: A Multi-Task Transformer for Robotic Manipulationcited by 551
  11. 2023MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generationcited by 536
  12. 2022Winoground: Probing Vision and Language Models for Visio-Linguistic Compositionalitycited by 529
38 more
  1. 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Modelscited by 513
  2. 2022MagicVideo: Efficient Video Generation With Latent Diffusion Modelscited by 465
  3. 2023Emergent Correspondence from Image Diffusioncited by 446
  4. 2021SLIP: Self-supervision meets Language-Image Pre-trainingcited by 436
  5. 2022Diffusion Art or Digital Forgery? Investigating Data Replication in Diffusion Modelscited by 424
  6. 2023SalUn: Empowering Machine Unlearning via Gradient-based Weight Saliency in Both Image Classification and Generationcited by 386
  7. 2023NLP Evaluation in trouble: On the Need to Measure LLM Data Contamination for each Benchmarkcited by 348
  8. 2023OBELICS: An Open Web-Scale Filtered Dataset of Interleaved Image-Text Documentscited by 316
  9. 2023Poisoning Web-Scale Training Datasets is Practicalcited by 290
  10. 2022Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scalecited by 262
  11. 2023Generating Images with Multimodal Language Modelscited by 260
  12. 2023Challenges and Applications of Large Language Modelscited by 254
  13. 2022Data Contamination: From Memorization to Exploitationcited by 236
  14. 2022Imagen Editor and EditBench: Advancing and Evaluating Text-Guided Image Inpaintingcited by 233
  15. 2022DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Modelscited by 230
  16. 2023To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Nowcited by 210
  17. 2023Multimodal C4: An Open, Billion-scale Corpus of Images Interleaved with Textcited by 208
  18. 2022Multimodal Contrastive Learning with LIMoE: the Language-Image Mixture of Expertscited by 203
  19. 2023The political ideology of conversational AI: Converging evidence on ChatGPT's pro-environmental, left-libertarian orientationcited by 189
  20. 2022MERLOT Reserve: Neural Script Knowledge through Vision and Language and Soundcited by 184
  21. 2022Fake it till you make it: Learning transferable representations from synthetic ImageNet clonescited by 172
  22. 2024Defensive Unlearning with Adversarial Training for Robust Concept Erasure in Diffusion Modelscited by 168
  23. 2023Sociotechnical Safety Evaluation of Generative AI Systemscited by 157
  24. 2023Debiasing Vision-Language Models via Biased Promptscited by 149
  25. 2024The PRISM Alignment Dataset: What Participatory, Representative and Individualised Human Feedback Reveals About the Subjective and Multicultural Alignment of Large Language Modelscited by 144
  26. 2022CM3: A Causal Masked Multimodal Model of the Internetcited by 142
  27. 2023Grounding Language Models to Images for Multimodal Inputs and Outputscited by 131
  28. 2023Social Biases through the Text-to-Image Generation Lenscited by 127
  29. 2024Black-Box Access is Insufficient for Rigorous AI Auditscited by 126
  30. 2022Retrieval-Augmented Multimodal Language Modelingcited by 125
  31. 2023Generative AIcited by 120
  32. 2022Vision-Language Pre-training: Basics, Recent Advances, and Future Trendscited by 117
  33. 2022Revisiting Weakly Supervised Pre-Training of Visual Perception Modelscited by 116
  34. 2022Foundations and Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questionscited by 116
  35. 2022VectorFusion: Text-to-SVG by Abstracting Pixel-Based Diffusion Modelscited by 112
  36. 2022A Prompt Array Keeps the Bias Away: Debiasing Vision-Language Models with Adversarial Learningcited by 104
  37. 2023DreamHuman: Animatable 3D Avatars from Textcited by 102
  38. 2022Quality Not Quantity: On the Interaction between Dataset Design and Robustness of CLIPcited by 97

50 most cited of 327 All 327 (JSON)