# OpenBias: Open-set Bias Detection in Text-to-Image Generative Models

arXiv: https://arxiv.org/abs/2404.07990 · HTML: https://mlsanity.com/paper/2404.07990/

## Key facts

- Cited by 61 papers in the map (data through 14 Oct 2026).
- Builds on 63 of its 77 references that are in the map.
- First posted on arXiv 11 Apr 2024.

## Builds on (most cited first, 30 of 63)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Improved Baselines with Visual Instruction Tuning](https://mlsanity.com/paper/2310.03744/) (2023)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision](https://mlsanity.com/paper/2102.03334/) (2021)
- [Alias-Free Generative Adversarial Networks](https://mlsanity.com/paper/2106.12423/) (2021)
- [Reproducible scaling laws for contrastive language-image learning](https://mlsanity.com/paper/2212.07143/) (2022)
- [ConceptNet 5.5: An Open Multilingual Graph of General Knowledge](https://mlsanity.com/paper/1612.03975/) (2016)

## Built on by (most cited first, 30 of 61)

- [HumaniBench: A Human-Centric Framework for Large Multimodal Models Evaluation](https://mlsanity.com/paper/2505.11454/) (2025)
- [A Large Scale Analysis of Gender Biases in Text-to-Image Generative Models](https://mlsanity.com/paper/2503.23398/) (2025)
- [Backdooring Bias (B2) into Stable Diffusion Models](https://mlsanity.com/paper/2406.15213/) (2024)
- [Replication in Visual Diffusion Models: A Survey and Outlook](https://mlsanity.com/paper/2408.00001/) (2024)
- [Where Culture Fades: Revealing the Cultural Gap in Text-to-Image Generation](https://mlsanity.com/paper/2511.17282/) (2025)
- [Culture in Action: Evaluating Text-to-Image Models through Social Activities](https://mlsanity.com/paper/2511.05681/) (2025)
- [Analyzing Quality, Bias, and Performance in Text-to-Image Generative Models](https://mlsanity.com/paper/2407.00138/) (2024)
- [Re-Imagining Multimodal Instruction Tuning: A Representation View](https://mlsanity.com/paper/2503.00723/) (2025)
- [Exploring Bias in over 100 Text-to-Image Generative Models](https://mlsanity.com/paper/2503.08012/) (2025)
- [OASIS Uncovers: High-Quality T2I Models, Same Old Stereotypes](https://mlsanity.com/paper/2501.00962/) (2025)
- [Organizing Unstructured Image Collections using Natural Language](https://mlsanity.com/paper/2410.05217/) (2024)
- [Trustworthy Text-to-Image Diffusion Models: A Timely and Focused Survey](https://mlsanity.com/paper/2409.18214/) (2024)
- [POET: Supporting Prompting Creativity and Personalization with Automated Expansion of Text-to-Image Generation](https://mlsanity.com/paper/2504.13392/) (2025)
- [GRADE: Quantifying Sample Diversity in Text-to-Image Models](https://mlsanity.com/paper/2410.22592/) (2024)
- [MAVias: Mitigate any Visual Bias](https://mlsanity.com/paper/2412.06632/) (2024)
- [Say My Name: a Model's Bias Discovery Framework](https://mlsanity.com/paper/2408.09570/) (2024)
- [Implicit Bias Injection Attacks against Text-to-Image Diffusion Models](https://mlsanity.com/paper/2504.01819/) (2025)
- [Classifier-to-Bias: Toward Unsupervised Automatic Bias Detection for Visual Classifiers](https://mlsanity.com/paper/2504.20902/) (2025)
- [DIMCIM: A Quantitative Evaluation Framework for Default-mode Diversity and Generalization in Text-to-Image Generative Models](https://mlsanity.com/paper/2506.05108/) (2025)
- [How I Met Your Bias: Investigating Bias Amplification in Diffusion Models](https://mlsanity.com/paper/2512.20233/) (2025)
- [Diffusing DeBias: Synthetic Bias Amplification for Model Debiasing](https://mlsanity.com/paper/2502.09564/) (2025)
- [Group-robust Machine Unlearning](https://mlsanity.com/paper/2503.09330/) (2025)
- [Multi-Group Proportional Representation for Text-to-Image Models](https://mlsanity.com/paper/2505.24023/) (2025)
- [Processing and acquisition traces in visual encoders: What does CLIP know about your camera?](https://mlsanity.com/paper/2508.10637/) (2025)
- [RespoDiff: Dual-Module Bottleneck Transformation for Responsible & Faithful T2I Generation](https://mlsanity.com/paper/2509.15257/) (2025)
- [GradBias: Unveiling Word Influence on Bias in Text-to-Image Generative Models](https://mlsanity.com/paper/2408.16700/) (2024)
- [Is What You Ask For What You Get? Investigating Concept Associations in Text-to-Image Models](https://mlsanity.com/paper/2410.04634/) (2024)
- [On the Fairness, Diversity and Reliability of Text-to-Image Generative Models](https://mlsanity.com/paper/2411.13981/) (2024)
- [GIFT: A Framework Towards Global Interpretable Faithful Textual Explanations of Vision Classifiers](https://mlsanity.com/paper/2411.15605/) (2024)
- [BiasConnect: Investigating Bias Interactions in Text-to-Image Models](https://mlsanity.com/paper/2503.09763/) (2025)

## Abstract (arXiv)

Text-to-image generative models are becoming increasingly popular and accessible to the general public. As these models see large-scale deployments, it is necessary to deeply investigate their safety and fairness to not disseminate and perpetuate any kind of biases. However, existing works focus on detecting closed sets of biases defined a priori, limiting the studies to well-known concepts. In this paper, we tackle the challenge of open-set bias detection in text-to-image generative models presenting OpenBias, a new pipeline that identifies and quantifies the severity of biases agnostically, without access to any precompiled set. OpenBias has three stages. In the first phase, we leverage a Large Language Model (LLM) to propose biases given a set of captions. Secondly, the target generative model produces images using the same set of captions. Lastly, a Vision Question Answering model recognizes the presence and extent of the previously proposed biases. We study the behavior of Stable Diffusion 1.5, 2, and XL emphasizing new biases, never investigated before. Via quantitative experiments, we demonstrate that OpenBias agrees with current closed-set bias detection methods and human judgement.