# TIBET: Identifying and Evaluating Biases in Text-to-Image Generative Models

arXiv: https://arxiv.org/abs/2312.01261 · HTML: https://mlsanity.com/paper/2312.01261/

## Key facts

- Cited by 52 papers in the map (data through 14 Oct 2026).
- Builds on 52 of its 76 references that are in the map.
- First posted on arXiv 3 Dec 2023.

## Builds on (most cited first, 30 of 52)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Chain-of-Thought Prompting Elicits Reasoning in Large Language Models](https://mlsanity.com/paper/2201.11903/) (2022)
- [VQA: Visual Question Answering](https://mlsanity.com/paper/1505.00468/) (2015)
- [Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings](https://mlsanity.com/paper/1607.06520/) (2016)
- [Counterfactual Fairness](https://mlsanity.com/paper/1703.06856/) (2017)
- [Network Dissection: Quantifying Interpretability of Deep Visual Representations](https://mlsanity.com/paper/1704.05796/) (2017)
- [Automatic Chain of Thought Prompting in Large Language Models](https://mlsanity.com/paper/2210.03493/) (2022)
- [MiniGPT-v2: large language model as a unified interface for vision-language multi-task learning](https://mlsanity.com/paper/2310.09478/) (2023)
- [Explaining Machine Learning Classifiers through Diverse Counterfactual Explanations](https://mlsanity.com/paper/1905.07697/) (2019)
- [Learning the Difference that Makes a Difference with Counterfactually-Augmented Data](https://mlsanity.com/paper/1909.12434/) (2019)
- [TIFA: Accurate and Interpretable Text-to-Image Faithfulness Evaluation with Question Answering](https://mlsanity.com/paper/2303.11897/) (2023)
- [DiffusionDB: A Large-scale Prompt Gallery Dataset for Text-to-Image Generative Models](https://mlsanity.com/paper/2210.14896/) (2022)
- [Measuring Bias in Contextualized Word Representations](https://mlsanity.com/paper/1906.07337/) (2019)
- [Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scale](https://mlsanity.com/paper/2211.03759/) (2022)
- [Towards Fairness in Visual Recognition: Effective Strategies for Bias Mitigation](https://mlsanity.com/paper/1911.11834/) (2019)
- [Women also Snowboard: Overcoming Bias in Captioning Models](https://mlsanity.com/paper/1803.09797/) (2018)
- [Break-A-Scene: Extracting Multiple Concepts from a Single Image](https://mlsanity.com/paper/2305.16311/) (2023)
- [DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Models](https://mlsanity.com/paper/2202.04053/) (2022)
- [Stable Bias: Analyzing Societal Representations in Diffusion Models](https://mlsanity.com/paper/2303.11408/) (2023)
- [Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Models](https://mlsanity.com/paper/2101.00288/) (2021)
- [Social Biases in NLP Models as Barriers for Persons with Disabilities](https://mlsanity.com/paper/2005.00813/) (2020)
- [Path-Specific Counterfactual Fairness](https://mlsanity.com/paper/1802.08139/) (2018)
- [Fair Diffusion: Instructing Text-to-Image Generation Models on Fairness](https://mlsanity.com/paper/2302.10893/) (2023)
- [Predictive Biases in Natural Language Processing Models: A Conceptual Framework and Overview](https://mlsanity.com/paper/1912.11078/) (2019)
- [REVISE: A Tool for Measuring and Mitigating Bias in Visual Datasets](https://mlsanity.com/paper/2004.07999/) (2020)
- [CausaLM: Causal Model Explanation Through Counterfactual Language Models](https://mlsanity.com/paper/2005.13407/) (2020)
- [Show and Tell: Lessons learned from the 2015 MSCOCO Image Captioning Challenge](https://mlsanity.com/paper/1609.06647/) (2016)
- [Mitigating Language-Dependent Ethnic Bias in BERT](https://mlsanity.com/paper/2109.05704/) (2021)
- [ChatGPT Asks, BLIP-2 Answers: Automatic Questioning Towards Enriched Visual Descriptions](https://mlsanity.com/paper/2303.06594/) (2023)

## Built on by (most cited first, 30 of 52)

- [Survey of Bias In Text-to-Image Generation: Definition, Evaluation, and Mitigation](https://mlsanity.com/paper/2404.01030/) (2024)
- [Stable Diffusion Exposed: Gender Bias from Prompt to Image](https://mlsanity.com/paper/2312.03027/) (2023)
- [FAIntbench: A Holistic and Precise Benchmark for Bias Evaluation in Text-to-Image Models](https://mlsanity.com/paper/2405.17814/) (2024)
- [BIGbench: A Unified Benchmark for Evaluating Multi-dimensional Social Biases in Text-to-Image Models](https://mlsanity.com/paper/2407.15240/) (2024)
- [Quantifying Bias in Text-to-Image Generative Models](https://mlsanity.com/paper/2312.13053/) (2023)
- [Multilingual Text-to-Image Generation Magnifies Gender Stereotypes and Prompt Engineering May Not Help You](https://mlsanity.com/paper/2401.16092/) (2024)
- [A Large Scale Analysis of Gender Biases in Text-to-Image Generative Models](https://mlsanity.com/paper/2503.23398/) (2025)
- [Fairness and Bias Mitigation in Computer Vision: A Survey](https://mlsanity.com/paper/2408.02464/) (2024)
- [Exploring Bias in over 100 Text-to-Image Generative Models](https://mlsanity.com/paper/2503.08012/) (2025)
- [Fair Generation without Unfair Distortions: Debiasing Text-to-Image Generation with Entanglement-Free Attention](https://mlsanity.com/paper/2506.13298/) (2025)
- [FairQueue: Rethinking Prompt Learning for Fair Text-to-Image Generation](https://mlsanity.com/paper/2410.18615/) (2024)
- [VersusDebias: Universal Zero-Shot Debiasing for Text-to-Image Models via SLM-Based Prompt Engineering and Generative Adversary](https://mlsanity.com/paper/2407.19524/) (2024)
- [Trustworthy Text-to-Image Diffusion Models: A Timely and Focused Survey](https://mlsanity.com/paper/2409.18214/) (2024)
- [POET: Supporting Prompting Creativity and Personalization with Automated Expansion of Text-to-Image Generation](https://mlsanity.com/paper/2504.13392/) (2025)
- [Implicit Bias Injection Attacks against Text-to-Image Diffusion Models](https://mlsanity.com/paper/2504.01819/) (2025)
- [Exposing Blindspots: Cultural Bias Evaluation in Generative Image Models](https://mlsanity.com/paper/2510.20042/) (2025)
- [Generated Bias: Auditing Internal Bias Dynamics of Text-To-Image Generative Models](https://mlsanity.com/paper/2410.07884/) (2024)
- [Multi-Group Proportional Representation for Text-to-Image Models](https://mlsanity.com/paper/2505.24023/) (2025)
- [Is What You Ask For What You Get? Investigating Concept Associations in Text-to-Image Models](https://mlsanity.com/paper/2410.04634/) (2024)
- [On the Fairness, Diversity and Reliability of Text-to-Image Generative Models](https://mlsanity.com/paper/2411.13981/) (2024)
- [BiasConnect: Investigating Bias Interactions in Text-to-Image Models](https://mlsanity.com/paper/2503.09763/) (2025)
- [Mitigate One, Skew Another? Tackling Intersectional Biases in Text-to-Image Models](https://mlsanity.com/paper/2505.17280/) (2025)
- [Hidden Bias in the Machine: Stereotypes in Text-to-Image Models](https://mlsanity.com/paper/2506.13780/) (2025)
- [Discovering Divergent Representations between Text-to-Image Models](https://mlsanity.com/paper/2509.08940/) (2025)
- [TrueSkin: Towards Fair and Accurate Skin Tone Recognition and Generation](https://mlsanity.com/paper/2509.10980/) (2025)
- [Multi-Axis Max@K Reinforcement Learning for Representative Diversity in Text-to-Image Generation](https://mlsanity.com/paper/2607.14962/) (2026)
- [AITTI: Learning Adaptive Inclusive Token for Text-to-Image Generation](https://mlsanity.com/paper/2406.12805/) (2024)
- [Synthetic History: Evaluating Visual Representations of the Past in Diffusion Models](https://mlsanity.com/paper/2505.17064/) (2025)
- [Prompting Away Stereotypes? Evaluating Bias in Text-to-Image Models for Occupations](https://mlsanity.com/paper/2509.00849/) (2025)
- [Automated Evaluation of Gender Bias Across 13 Large Multimodal Models](https://mlsanity.com/paper/2509.07050/) (2025)

## Abstract (arXiv)

Text-to-Image (TTI) generative models have shown great progress in the past few years in terms of their ability to generate complex and high-quality imagery. At the same time, these models have been shown to suffer from harmful biases, including exaggerated societal biases (e.g., gender, ethnicity), as well as incidental correlations that limit such a model's ability to generate more diverse imagery. In this paper, we propose a general approach to study and quantify a broad spectrum of biases, for any TTI model and for any prompt, using counterfactual reasoning. Unlike other works that evaluate generated images on a predefined set of bias axes, our approach automatically identifies potential biases that might be relevant to the given prompt, and measures those biases. In addition, we complement quantitative scores with post-hoc explanations in terms of semantic concepts in the images generated. We show that our method is uniquely capable of explaining complex multi-dimensional biases through semantic concepts, as well as the intersectionality between different biases for any given prompt. We perform extensive user studies to illustrate that the results of our method and analysis are consistent with human judgements.