mlsanityData to 7 Oct 2026

T2ISafety: Benchmark for Assessing Fairness, Toxicity, and Privacy in Image Generation

arXiv 2501.12612 ↗First posted 22 Jan 2025cs.CL cs.CRcited by 47

Not on the map. Why?

Abstract (arXiv)

Text-to-image (T2I) models have rapidly advanced, enabling the generation of high-quality images from text prompts across various domains. However, these models present notable safety concerns, including the risk of generating harmful, biased, or private content. Current research on assessing T2I safety remains in its early stages. While some efforts have been made to evaluate models on specific safety dimensions, many critical risks remain unexplored. To address this gap, we introduce T2ISafety, a safety benchmark that evaluates T2I models across three key domains: toxicity, fairness, and bias. We build a detailed hierarchy of 12 tasks and 44 categories based on these three domains, and meticulously collect 70K corresponding prompts. Based on this taxonomy and prompt set, we build a large-scale T2I dataset with 68K manually annotated images and train an evaluator capable of detecting critical risks that previous work has failed to identify, including risks that even ultra-large proprietary models like GPTs cannot correctly detect. We evaluate 12 prominent diffusion models on T2ISafety and reveal several concerns including persistent issues with racial fairness, a tendency to generate toxic content, and significant variation in privacy protection across the models, even with defense methods like concept erasing. Data and evaluator are released under https://github.com/adwardlee/t2i_safety.

Key facts. Cited by 47 papers in the map (data through 7 Oct 2026). Builds on 69 of its 89 references that are in the map. First posted on arXiv 22 Jan 2025.

Builds on

69 of 89 references.

References · 69

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  4. 2023Visual Instruction Tuningcited by 7,792
  5. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  6. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  7. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  8. 2022Scalable Diffusion Models with Transformerscited by 5,931
  9. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  10. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  11. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
  12. 2023Qwen Technical Reportcited by 3,273
38 more
  1. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  2. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,534
  3. 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
  4. 2024Mixtral of Expertscited by 1,785
  5. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  6. 2023Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generationcited by 950
  7. 2023A Survey on Multimodal Large Language Modelscited by 898
  8. 2023PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesiscited by 854
  9. 2024Autoregressive Model Beats Diffusion: Llama for Scalable Image Generationcited by 760
  10. 2023Adversarial Diffusion Distillationcited by 741
  11. 2024Show-o: One Single Transformer to Unify Multimodal Understanding and Generationcited by 734
  12. 2023Erasing Concepts from Diffusion Modelscited by 530
  13. 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Modelscited by 501
  14. 2024Evaluating Text-to-Visual Generation with Image-to-Text Generationcited by 465
  15. 2023TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answeringcited by 416
  16. 2023Unified Concept Editing in Diffusion Modelscited by 377
  17. 2022DiffusionDB: A Large-scale Prompt Gallery Dataset for Text-to-Image Generative Modelscited by 376
  18. 2023InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuningcited by 343
  19. 2024InternLM-XComposer2: Mastering Free-form Text-Image Composition and Comprehension in Vision-Language Large Modelcited by 326
  20. 2024What matters when building vision-language models?cited by 321
  21. 2024MACE: Mass Concept Erasure in Diffusion Modelscited by 298
  22. 2024Hunyuan-DiT: A Powerful Multi-Resolution Diffusion Transformer with Fine-Grained Chinese Understandingcited by 294
  23. 2024VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generationcited by 290
  24. 2024MM-LLMs: Recent Advances in MultiModal Large Language Modelscited by 277
  25. 2022Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scalecited by 262
  26. 2024SDXL-Lightning: Progressive Adversarial Diffusion Distillationcited by 258
  27. 2022Red-Teaming the Stable Diffusion Safety Filtercited by 257
  28. 2024Playground v2.5: Three Insights towards Enhancing Aesthetic Quality in Text-to-Image Generationcited by 249
  29. 2022DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Modelscited by 230
  30. 2023Camels in a Changing Climate: Enhancing LM Adaptation with Tulu 2cited by 227
  31. 2024SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Modelscited by 225
  32. 2023JourneyDB: A Benchmark for Generative Image Understandingcited by 203
  33. 2023SneakyPrompt: Jailbreaking Text-to-image Generative Modelscited by 175
  34. 2023Unsafe Diffusion: On the Generation of Unsafe Images and Hateful Memes From Text-To-Image Modelscited by 171
  35. 2023LAMM: Language-Assisted Multi-Modal Instruction-Tuning Dataset, Framework, and Benchmarkcited by 169
  36. 2023Holistic Evaluation of Text-To-Image Modelscited by 160
  37. 2022Can Machines Help Us Answering Question 16 in Datasheets, and In Turn Reflecting on Inappropriate Content?cited by 150
  38. 2024VidProM: A Million-scale Real Prompt-Gallery Dataset for Text-to-Video Diffusion Modelscited by 124

50 most cited of 69 All 69 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 21; 2026: 26.

47 papers.

Sort:

By area:

  1. 2025RoboRefer: Towards Spatial Referring with Reasoning in Vision-Language Models for Roboticscited by 125
  2. 2025Layer-Aware Representation Filtering: Purifying Finetuning Data to Preserve LLM Safety Alignmentcited by 24
  3. 2025Visual Contextual Attack: Jailbreaking MLLMs with Image-Driven Context Injectioncited by 23
  4. 2025T2I-RiskyPrompt: A Benchmark for Safety Evaluation, Attack, and Defense on Text-to-Image Modelcited by 16
  5. 2025NavigateDiff: Visual Predictors are Zero-Shot Navigation Assistantscited by 11
  6. 2025Erasing Concepts, Steering Generations: A Comprehensive Survey of Concept Suppressioncited by 10
  7. 2025Response Attack: Exploiting Contextual Priming to Jailbreak Large Language Modelscited by 10
  8. 2025Multimodal LLMs as Customized Reward Models for Text-to-Image Generationcited by 9
  9. 2026GRP-Obliteration: Unaligning LLMs With a Single Unlabeled Promptcited by 6
  10. 2026Towards Policy-Adaptive Image Guardrail: Benchmark and Methodcited by 6
  11. 2025Trade-offs in Image Generation: How Do Different Dimensions Interact?cited by 3
  12. 2025Toxicity in Online Platforms and AI Systems: A Survey of Needs, Challenges, Mitigations, and Future Directionscited by 3
35 more
  1. 2026Conditioned Activation Transport for T2I Safety Steeringcited by 3
  2. 2026On the Adversarial Robustness of Multimodal LLM Judgescited by 3
  3. 2025Self-adaptive Dataset Construction for Real-World Multimodal Safety Scenarioscited by 2
  4. 2025FAME: Fairness-aware Attention-modulated Video Editingcited by 2
  5. 2025SafeEditor: Unified MLLM for Efficient Post-hoc T2I Safety Editingcited by 2
  6. 2025SCALEX: Scalable Concept and Latent Exploration for Diffusion Modelscited by 2
  7. 2025SPQR: A Multi-Dimensional Benchmark for Safety Alignment under Benign Model Adaptationcited by 2
  8. 2025Contextual Image Attack: How Visual Context Exposes Multimodal Safety Vulnerabilitiescited by 2
  9. 2025ProGuard: Towards Proactive Multimodal Safeguardcited by 2
  10. 2025Inference Time Debiasing Concepts in Diffusion Modelscited by 1
  11. 2025The Plot Twist: Jailbreaking Unified Multimodal Models with a Three-Act NarrativeAttackcited by 1
  12. 2025CompAgent: An Agentic Framework for Visual Compliance Verificationcited by 1
  13. 2026Bias at the End of the Scorecited by 1
  14. 2026HoloFair: Unified T2I Fairness Evaluation and Fair-GRPO Debiasingcited by 1
  15. 2026Innocent Panels, Hateful Stories: Evaluating and Detecting Hateful Intent in Multi-Turn Visual Story Generationcited by 1
  16. 2026What Current AI Benchmarks Leave Unmeasured: Modality, Search, Citations, and Implications (for Safety Evaluations)cited by 1
  17. 2025A Framework for Benchmarking Fairness-Utility Trade-offs in Text-to-Image Models via Pareto Frontierscited by 0
  18. 2026Consistency-Preserving Concept Erasure via Unsafe-Safe Pairing and Directional Fisher-weighted Adaptationcited by 0
  19. 2026Forgetting is Competition: Rethinking Unlearning as Representation Interference in Diffusion Modelscited by 0
  20. 2026Na"ive Exposure of Generative AI Capabilities Undermines Deepfake Detectioncited by 0
  21. 2026Does Unification Come at a Cost? Uni-SafeBench: A Safety Benchmark for Unified Multimodal Large Modelscited by 0
  22. 2026EDGE-Shield: Efficient Denoising-staGE Shield for Violative Content Filtering via Scalable Reference-Based Matchingcited by 0
  23. 2026When Safe Concepts Become Unsafe: Multi-Concept Compositional Vulnerabilities in Text-to-Image Modelscited by 0
  24. 2026Operationalizing Fairness in Text-to-Image Models: A Survey of Bias, Fairness Audits and Mitigation Strategiescited by 0
  25. 2026RedEdit: Agentic Red-Teaming of Image Safety Classifiers via MCTS-Guided Photo-Editingcited by 0
  26. 2026From Celebrities to Anyone: Characterizing AI Nudification Content, Technology, and Community Dynamics on 4chancited by 0
  27. 2026Harm is not Universal: Community-Specific Toxicity Detection is Urgently Neededcited by 0
  28. 2026GuardPaint:SpeculativeSafetyDecodingforText-to-ImageGenerationcited by 0
  29. 2026Multi2AV-Safety: Benchmarking Safety in Multimodal-to-Audio-Video Generationcited by 0
  30. 2026SafeAtlas-VL: Beyond Binary Multimodal Safety with Large-Scale Data and Guard Modelscited by 0
  31. 2026Not Safe for All: Auditing the Dialect Penalty in Text-to-Image Safety Pipelinescited by 0
  32. 2026AV-SafetyBench: A Safety Benchmark for Text-to-Audio-Video Generationcited by 0
  33. 2026IMPLICIT-Bench: Measuring Implicit Bias in Text-to-Image Models under Neutral Promptscited by 0
  34. 2026InGuard: Towards Generalized Inner Guardrail for Safe Text-to-Image Generationcited by 0
  35. 2026Steering Fields: Adaptive Vector Fields for Safe Image Generation and Beyondcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025The Path to Reconciling Quality and Safety in Text-to-Image Generation: Dataset, Method, and Evaluation
  2. 2025OVERT: A Benchmark for Over-Refusal Evaluation on Text-to-Image Models
  3. 2026SafeGen-Bench: Benchmarking Safety in Image-Conditioned Text-to-Video Generation
  4. 2025GenBreak: Red Teaming Text-to-Image Generators Using Large Language Models
  5. 2026DTVI: Dual-Stage Textual and Visual Intervention for Safe Text-to-Image Generation
  6. 2023Evaluating the Robustness of Text-to-image Diffusion Models against Real-world Attacks
  7. 2024AlignGuard: Scalable Safety Alignment for Text-to-Image Generation
  8. 2024Trustworthy Text-to-Image Diffusion Models: A Timely and Focused Survey