mlsanityData to 7 Oct 2026

Whose View of Safety? A Deep DIVE Dataset for Pluralistic Alignment of Text-to-Image Models

arXiv 2507.13383 ↗First posted 15 Jul 2025cs.LG cs.AI cs.CVcited by 11

On the map: LLM alignment / RLHF / preference optimisation → Safety alignment and jailbreak attacks

Abstract (arXiv)

Current text-to-image (T2I) models often fail to account for diverse human experiences, leading to misaligned systems. We advocate for pluralistic alignment, where an AI understands and is steerable towards diverse, and often conflicting, human values. Our work provides three core contributions to achieve this in T2I models. First, we introduce a novel dataset for Diverse Intersectional Visual Evaluation (DIVE) -- the first multimodal dataset for pluralistic alignment. It enable deep alignment to diverse safety perspectives through a large pool of demographically intersectional human raters who provided extensive feedback across 1000 prompts, with high replication, capturing nuanced safety perceptions. Second, we empirically confirm demographics as a crucial proxy for diverse viewpoints in this domain, revealing significant, context-dependent differences in harm perception that diverge from conventional evaluations. Finally, we discuss implications for building aligned T2I models, including efficient data collection strategies, LLM judgment capabilities, and model steerability towards diverse perspectives. This research offers foundational tools for more equitable and aligned T2I systems. Content Warning: The paper includes sensitive content that may be harmful.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Placed in Safety alignment and jailbreak attacks, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 41 of its 64 references that are in the map. First posted on arXiv 15 Jul 2025.

Builds on

41 of 64 references.

References · 41

  1. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,393
  2. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,500
  3. 2024A Survey on LLM-as-a-Judgecited by 1,384
  4. 2021Ethical and social risks of harm from Language Modelscited by 957
  5. 2023Whose Opinions Do Language Models Reflect?cited by 759
  6. 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Modelscited by 501
  7. 2021Dynabench: Rethinking Benchmarking in NLPcited by 447
  8. 2021Dealing with Disagreements: Looking Beyond the Majority Vote in Subjective Annotationscited by 306
  9. 2022Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scalecited by 262
  10. 2021Annotators with Attitudes: How Annotator Beliefs And Identities Bias Toxic Language Detectioncited by 258
  11. 2024LLMs instead of Human Judges? A Large Scale Empirical Study across 20 NLP Evaluation Taskscited by 236
  12. 2024Personalizing Reinforcement Learning from Human Feedback with Variational Preference Learningcited by 147
29 more
  1. 2021Bias Out-of-the-Box: An Empirical Analysis of Intersectional Occupational Biases in Popular Generative Language Modelscited by 141
  2. 2021On Releasing Annotator-Level Labels and Information in Datasetscited by 123
  3. 2024MaxMin-RLHF: Alignment with Diverse Human Preferencescited by 121
  4. 2022Jury Learning: Integrating Dissenting Voices into Machine Learning Modelscited by 115
  5. 2023Value Kaleidoscope: Engaging AI with Pluralistic Human Values, Rights, and Dutiescited by 97
  6. 2023DICES Dataset: Diversity in Conversational AI Evaluation for Safetycited by 86
  7. 2021Anticipating Safety Issues in E2E Conversational AI: Framework and Toolingcited by 77
  8. 2024PERSONA: A Reproducible Testbed for Pluralistic Alignmentcited by 71
  9. 2021Designing Toxic Content Classification for a Diversity of Perspectivescited by 63
  10. 2023When Do Annotator Demographics Matter? Measuring the Influence of Annotator Demographics with the POPQUORN Datasetcited by 51
  11. 2021ConvAbuse: Data, Analysis, and Benchmarks for Nuanced Abuse Detection in Conversational AIcited by 46
  12. 2023Inspecting the Geographical Representativeness of Images from Text-to-Image Modelscited by 44
  13. 2025Cultivating Pluralism In Algorithmic Monoculture: The Community Alignment Datasetcited by 32
  14. 2025Beyond Demographics: Fine-tuning Large Language Models to Predict Individuals' Subjective Text Perceptionscited by 31
  15. 2021Cross-replication Reliability -- An Empirical Approach to Interpreting Inter-rater Reliabilitycited by 30
  16. 2025Value Profiles for Encoding Human Variationcited by 29
  17. 2025ShieldGemma 2: Robust and Tractable Image Content Moderationcited by 27
  18. 2023Disentangling Perceptions of Offensiveness: Cultural and Moral Correlatescited by 26
  19. 2023How Far Can We Extract Diverse Perspectives from Large Language Models?cited by 22
  20. 2023Concrete Problems in AI Safety, Revisitedcited by 21
  21. 2024Adversarial Nibbler: An Open Red-Teaming Method for Identifying Diverse Harms in Text-to-Image Generationcited by 20
  22. 2023GRASP: A Disagreement Analysis Framework to Assess Group Associations in Perspectivescited by 16
  23. 2025CulturalFrames: Assessing Cultural Expectation Alignment in Text-to-Image Models and Evaluation Metricscited by 14
  24. 2025Collab: Controlled Decoding using Mixture of Agents for LLM Alignmentcited by 13
  25. 2023The Reasonable Effectiveness of Diverse Evaluation Datacited by 10
  26. 2023Beyond the ML Model: Applying Safety Engineering Frameworks to Text-to-Image Developmentcited by 9
  27. 2023Adversarial Nibbler: A Data-Centric Challenge for Improving the Safety of Text-to-Image Modelscited by 7
  28. 2024LLMs in the Loop: Leveraging Large Language Model Annotations for Active Learning in Low-Resource Languagescited by 7
  29. 2023Lived Experience Matters: Automatic Detection of Stigma on Social Media Toward People Who Use Substancescited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 8.

11 papers.

  1. 2026Computational Hermeneutics: Evaluating generative AI as a cultural technologycited by 7
  2. 2026PluriHarms: Benchmarking the Full Spectrum of Human Judgments on AI Harmcited by 5
  3. 2025Murphys Laws of AI Alignment: Why the Gap Always Winscited by 4
  4. 2025Reward Models are Metrics in a Trench Coatcited by 2
  5. 2026A Unified Framework to Quantify Cultural Intelligence of AIcited by 2
  6. 2026ProEval: Proactive Failure Discovery and Efficient Performance Estimation for Generative AI Evaluationcited by 2
  7. 2026Going PLACES: Participatory Localized Red Teaming for Text-to-Image Safety in the Global Southcited by 1
  8. 2025When Cars Have Stereotypes: Auditing Demographic Bias in Objects from Text-to-Image Modelscited by 0
  9. 2026Learning Who Disagrees: Demographic Importance Weighting for Modeling Annotator Distributions with DiADEMcited by 0
  10. 2026Sympathetic Framing: Evaluating AI Alignment across Sociodemographic Groupscited by 0
  11. 2026Multi-Dimensional Comparative Scale Construction for Efficient Personalized Subjective Judgment in High-Traffic Applicationscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026When Seeing Overrides Knowing: Visual Dominance and Deferral-Based Method for Personalized Safety in VLMs
  2. 2026From Local Evidence to Safety Verdicts: Causal Tracing in Vision-Language Models
  3. 2026When Helpers Become Hazards: A Benchmark for Analyzing Multimodal LLM-Powered Safety in Daily Life
  4. 2024SafeSora: Towards Safety Alignment of Text2Video Generation via a Human Preference Dataset
  5. 2026Pluralis v0.1: Towards a Multicultural, Multimodal, Multilingual Benchmark for AI Risk and Reliability
  6. 2025Attention Misses Visual Risk: Risk-Adaptive Steering for Multimodal Safety Alignment
  7. 2026Visual Self-Fulfilling Alignment: Shaping Safety-Oriented Personas via Threat-Related Images
  8. 2025Personalized Safety Alignment for Text-to-Image Diffusion Models