mlsanityData to 7 Oct 2026

Exploring Social Bias in Downstream Applications of Text-to-Image Foundation Models

arXiv 2312.10065 ↗First posted 5 Dec 2023cs.CY cs.AIcited by 7

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

Text-to-image diffusion models have been adopted into key commercial workflows, such as art generation and image editing. Characterising the implicit social biases they exhibit, such as gender and racial stereotypes, is a necessary first step in avoiding discriminatory outcomes. While existing studies on social bias focus on image generation, the biases exhibited in alternate applications of diffusion-based foundation models remain under-explored. We propose methods that use synthetic images to probe two applications of diffusion models, image editing and classification, for social bias. Using our methodology, we uncover meaningful and significant inter-sectional social biases in textitStable Diffusion, a state-of-the-art open-source text-to-image model. Our findings caution against the uninformed adoption of text-to-image foundation models for downstream tasks and services.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 25 of its 43 references that are in the map. First posted on arXiv 5 Dec 2023.

Builds on

25 of 43 references.

Closest ancestors · 5

Same direction, newest first

  1. 2023InstructEdit: Improving Automatic Masks for Diffusion-based Image Editing With User Instructions29 May
  2. 2023A Tale of Two Features: Stable Diffusion Complements DINO for Zero-Shot Semantic Correspondence24 May
  3. 2023Analyzing Bias in Diffusion-based Face Generation Models10 May
  4. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr
  5. 2021Palette: Image-to-Image Diffusion Models10 Nov

Other references · 20

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  3. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  4. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,736
  5. 2023Stable Bias: Analyzing Societal Representations in Diffusion Modelscited by 225
  6. 2021Evaluating Deep Neural Networks Trained on Clinical Images in Dermatology with the Fitzpatrick 17k Datasetcited by 184
  7. 2020Detecting Emergent Intersectional Biases: Contextualized Word Embeddings Contain a Distribution of Human-like Biasescited by 167
  8. 2023Social Biases through the Text-to-Image Generation Lenscited by 126
  9. 2023Diffusion Models for Open-Vocabulary Segmentationcited by 63
  10. 2020Exploring Racial Bias within Face Recognition via per-subject Adversarially-Enabled Data Augmentationcited by 45
  11. 2022American == White in Multimodal Language-and-Image AIcited by 39
  12. 2023T2IAT: Measuring Valence and Stereotypical Biases in Text-to-Image Generationcited by 39
8 more
  1. 2022Towards Transparency in Dermatology Image Datasets with Skin Tone Annotations by Experts, Crowds, and an Algorithmcited by 35
  2. 2019Estimating Skin Tone and Effects on Classification Performance in Dermatology Datasetscited by 29
  3. 2023Benchmarking Algorithmic Bias in Face Recognition: An Experimental Approach Using Synthetic Faces and Human Evaluationcited by 25
  4. 2021Intersectional Bias in Causal Language Modelscited by 24
  5. 2021Measuring Fairness with Biased Rulers: A Survey on Quantifying Biases in Pretrained Language Modelscited by 20
  6. 2023Evaluating Biased Attitude Associations of Language Models in an Intersectional Contextcited by 20
  7. 2017Human Skin Detection Using RGB, HSV and YCbCr Color Modelscited by 16
  8. 2021WordBias: An Interactive Visual Tool for Discovering Intersectional Biases Encoded in Word Embeddingscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 5; 2025: 1; 2026: 1.

7 papers.

  1. 2024A Comprehensive Survey of Foundation Models in Medicinecited by 56
  2. 2026Large Language Model Reasoning Failurescited by 20
  3. 2024Harm Amplification in Text-to-Image Modelscited by 14
  4. 2024In-Context Learning May Not Elicit Trustworthy Reasoning: A-Not-B Errors in Pretrained Language Modelscited by 8
  5. 2024KAHANI: Culturally-Nuanced Visual Storytelling Tool for Non-Western Culturescited by 4
  6. 2025An Exploration of Default Images in Text-to-Image Generationcited by 1
  7. 2024Building an Ethical and Trustworthy Biomedical AI Ecosystem for the Translational and Clinical Integration of Foundational Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Generated Bias: Auditing Internal Bias Dynamics of Text-To-Image Generative Models
  2. 2025Hidden Bias in the Machine: Stereotypes in Text-to-Image Models
  3. 2024New Job, New Gender? Measuring the Social Bias in Image Generation Models
  4. 2023Finetuning Text-to-Image Diffusion Models for Fairness
  5. 2023Stable Diffusion Exposed: Gender Bias from Prompt to Image
  6. 2024MIST: Mitigating Intersectional Bias with Disentangled Cross-Attention Editing in Text-to-Image Diffusion Models
  7. 2025Dissecting and Mitigating Diffusion Bias via Mechanistic Interpretability
  8. 2024Image-Perfect Imperfections: Safety, Bias, and Authenticity in the Shadow of Text-To-Image Model Evolution