mlsanityData to 14 Oct 2026

A Systematic Review of Open Datasets Used in Text-to-Image (T2I) Gen AI Model Safety

arXiv 2503.00020 ↗First posted 23 Feb 2025cs.CL cs.AI cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

Novel research aimed at text-to-image (T2I) generative AI safety often relies on publicly available datasets for training and evaluation, making the quality and composition of these datasets crucial. This paper presents a comprehensive review of the key datasets used in the T2I research, detailing their collection methods, compositions, semantic and syntactic diversity of prompts and the quality, coverage, and distribution of harm types in the datasets. By highlighting the strengths and limitations of the datasets, this study enables researchers to find the most relevant datasets for a use case, critically assess the downstream impacts of their work given the dataset distribution, particularly regarding model safety and ethical considerations, and also identify the gaps in dataset coverage and quality that future research may address.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 32 of its 49 references that are in the map. First posted on arXiv 23 Feb 2025.

Builds on

32 of 49 references.

References · 32

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  3. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  4. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  5. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  6. 2021On the Opportunities and Risks of Foundation Modelscited by 4,033
  7. 2021Ethical and social risks of harm from Language Modelscited by 966
  8. 2023Erasing Concepts from Diffusion Modelscited by 539
  9. 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Modelscited by 513
  10. 2021Multimodal datasets: misogyny, pornography, and malignant stereotypescited by 327
  11. 2020Large image datasets: A pyrrhic win for computer vision?cited by 237
  12. 2023MMA-Diffusion: MultiModal Attack on Diffusion Modelscited by 194
20 more
  1. 2023SneakyPrompt: Jailbreaking Text-to-image Generative Modelscited by 189
  2. 2021Process for Adapting Language Models to Society (PALMS) with Values-Targeted Datasetscited by 181
  3. 2023Prompting4Debugging: Red-Teaming Text-to-Image Diffusion Models by Finding Problematic Promptscited by 171
  4. 2019Towards Fairer Datasets: Filtering and Balancing the Distribution of the People Subtree in the ImageNet Hierarchycited by 156
  5. 2020Towards Accountability for Machine Learning Datasets: Practices from Software Engineering and Infrastructurecited by 138
  6. 2020REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasetscited by 136
  7. 2023#InsTag: Instruction Tagging for Analyzing Supervised Fine-tuning of Large Language Modelscited by 109
  8. 2019Evaluating the State-of-the-Art of End-to-End Natural Language Generation: The E2E NLG Challengecited by 86
  9. 2024SafetyPrompts: a Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safetycited by 77
  10. 2024Datasets for Large Language Models: A Comprehensive Surveycited by 71
  11. 2024Latent Guard: a Safety Framework for Text-to-image Generationcited by 71
  12. 2024Inadequacies of Large Language Model Benchmarks in the Era of Generative Artificial Intelligencecited by 68
  13. 2024SafeGen: Mitigating Sexually Explicit Content Generation in Text-to-Image Modelscited by 66
  14. 2022Gender Artifacts in Visual Datasetscited by 38
  15. 2024AI Risk Categorization Decoded (AIR 2024): From Government Regulations to Corporate Policiescited by 37
  16. 2024ART: Automatic Red-teaming for Text-to-Image Models to Protect Benign Userscited by 35
  17. 2024A Survey on Multimodal Benchmarks: In the Era of Large AI Modelscited by 27
  18. 2024A Comprehensive Survey of Direct Preference Optimization: Datasets, Theories, Variants, and Applicationscited by 27
  19. 2024Adversarial Nibbler: An Open Red-Teaming Method for Identifying Diverse Harms in Text-to-Image Generationcited by 20
  20. 2023ParaAMR: A Large-Scale Syntactically Diverse Paraphrase Dataset by AMR Back-Translationcited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025OVERT: A Benchmark for Over-Refusal Evaluation on Text-to-Image Modelscited by 5
  2. 2026Beyond wheelchairs and blindfolds: Investigating disability stereotypes in T2I models with INCLUDE-BENCHcited by 0