mlsanityData to 14 Oct 2026

The Bias Amplification Paradox in Text-to-Image Generation

arXiv 2308.00755 ↗First posted 1 Aug 2023cs.LG cs.CL cs.CVcited by 70

Not on the map. Why?

Cited by 26 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Bias amplification is a phenomenon in which models exacerbate biases or stereotypes present in the training data. In this paper, we study bias amplification in the text-to-image domain using Stable Diffusion by comparing gender ratios in training vs. generated images. We find that the model appears to amplify gender-occupation biases found in the training data (LAION) considerably. However, we discover that amplification can be largely attributed to discrepancies between training captions and model prompts. For example, an inherent difference is that captions from the training data often contain explicit gender information while our prompts do not, which leads to a distribution shift and consequently inflates bias measures. Once we account for distributional differences between texts used for training and generation when evaluating amplification, we observe that amplification decreases drastically. Our findings illustrate the challenges of comparing biases in models and their training data, and highlight confounding factors that impact analyses.

Key facts. Cited by 70 papers in the map (data through 14 Oct 2026). Builds on 39 of its 56 references that are in the map. First posted on arXiv 1 Aug 2023.

Builds on

39 of 56 references.

References · 39

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  4. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  5. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  6. 2020The Pile: An 800GB Dataset of Diverse Text for Language Modelingcited by 2,411
  7. 2023Pythia: A Suite for Analyzing Large Language Models Across Training and Scalingcited by 1,786
  8. 2023Extracting Training Data from Diffusion Modelscited by 732
  9. 2018Gender Bias in Coreference Resolution: Evaluation and Debiasing Methodscited by 659
  10. 2021Documenting Large Webtext Corpora: A Case Study on the Colossal Clean Crawled Corpuscited by 492
  11. 2022Large Language Models Struggle to Learn Long-Tail Knowledgecited by 458
  12. 2018Gender Bias in Coreference Resolutioncited by 442
27 more
  1. 2017Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraintscited by 420
  2. 2019Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Settingcited by 349
  3. 2021Multimodal datasets: misogyny, pornography, and malignant stereotypescited by 327
  4. 2022Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scalecited by 262
  5. 2022DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Modelscited by 230
  6. 2023Stable Bias: Analyzing Societal Representations in Diffusion Modelscited by 226
  7. 2023A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicitycited by 182
  8. 2018Adversarial Removal of Demographic Attributes from Text Datacited by 170
  9. 2023Fair Diffusion: Instructing Text-to-Image Generation Models on Fairnesscited by 151
  10. 2021Bias Out-of-the-Box: An Empirical Analysis of Intersectional Occupational Biases in Popular Generative Language Modelscited by 141
  11. 2023Social Biases through the Text-to-Image Generation Lenscited by 127
  12. 2022How well can Text-to-Image Generative Models understand Ethical Natural Language Interventions?cited by 92
  13. 2022A Systematic Study of Bias Amplificationcited by 65
  14. 2022Measuring Causal Effects of Data Statistics on Language Model's `Factual' Predictionscited by 57
  15. 2021Directional Bias Amplificationcited by 54
  16. 2023Uncurated Image-Text Datasets: Shedding Light on Demographic Biascited by 52
  17. 2023Inspecting the Geographical Representativeness of Images from Text-to-Image Modelscited by 44
  18. 2022Quantifying Social Biases Using Templates is Unreliablecited by 39
  19. 2023T2IAT: Measuring Valence and Stereotypical Biases in Text-to-Image Generationcited by 39
  20. 2022Quantifying Societal Bias Amplification in Image Captioningcited by 37
  21. 2022The Tail Wagging the Dog: Dataset Construction Biases of Social Bias Benchmarkscited by 31
  22. 2023Vision-Language Models Performing Zero-Shot Tasks Exhibit Gender-based Disparitiescited by 31
  23. 2022Underspecification in Scene Description-to-Depiction Taskscited by 30
  24. 2023A Friendly Face: Do Text-to-Image Systems Rely on Stereotypes when the Input is Under-Specified?cited by 25
  25. 2023Auditing Gender Presentation Differences in Text-to-Image Modelscited by 22
  26. 2021Causal Direction of Data Collection Matters: Implications of Causal and Anticausal Learning for NLPcited by 21
  27. 2022Write It Like You See It: Detectable Differences in Clinical Notes By Race Lead To Differential Model Recommendationscited by 20

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 7; 2024: 16; 2025: 28; 2026: 19.

70 papers.

Sort:

By area:

  1. 2023Unified Concept Editing in Diffusion Modelscited by 388
  2. 2023What's In My Big Data?cited by 129
  3. 2023Dataset Diffusion: Diffusion-based Synthetic Dataset Generation for Pixel-Level Semantic Segmentationcited by 112
  4. 2023Finetuning Text-to-Image Diffusion Models for Fairnesscited by 92
  5. 2024No "Zero-Shot" Without Exponential Data: Pretraining Concept Frequency Determines Multimodal Model Performancecited by 75
  6. 2023TIBET: Identifying and Evaluating Biases in Text-to-Image Generative Modelscited by 52
  7. 2025T2ISafety: Benchmark for Assessing Fairness, Toxicity, and Privacy in Image Generationcited by 48
  8. 2024Securing Large Language Models: Addressing Bias, Misinformation, and Prompt Attackscited by 26
  9. 2023Stable Diffusion Exposed: Gender Bias from Prompt to Imagecited by 25
  10. 2024Would Deep Generative Models Amplify Bias in Future Models?cited by 24
  11. 2025On Linear Representations and Pretraining Data Frequency in Language Modelscited by 23
  12. 2024Rethinking Training for De-biasing Text-to-Image Generation: Unlocking the Potential of Stable Diffusioncited by 22
38 more
  1. 2026Large Language Model Reasoning Failurescited by 21
  2. 2024Eureka: Evaluating and Understanding Large Foundation Modelscited by 18
  3. 2023Quantifying Bias in Text-to-Image Generative Modelscited by 16
  4. 2024Revealing and Reducing Gender Biases in Vision and Language Assistants (VLAs)cited by 15
  5. 2025A Large Scale Analysis of Gender Biases in Text-to-Image Generative Modelscited by 14
  6. 2025Where Culture Fades: Revealing the Cultural Gap in Text-to-Image Generationcited by 11
  7. 2025Exploring Bias in over 100 Text-to-Image Generative Modelscited by 9
  8. 2024FairGen: Enhancing Fairness in Text-to-Image Diffusion Models via Self-Discovering Latent Directionscited by 8
  9. 2025OASIS Uncovers: High-Quality T2I Models, Same Old Stereotypescited by 8
  10. 2025CuRe: Cultural Gaps in the Long Tail of Text-to-Image Systemscited by 8
  11. 2024Image-Perfect Imperfections: Safety, Bias, and Authenticity in the Shadow of Text-To-Image Model Evolutioncited by 7
  12. 2024GRADE: Quantifying Sample Diversity in Text-to-Image Modelscited by 5
  13. 2025How to Train your Text-to-Image Model: Evaluating Design Choices for Synthetic Training Captionscited by 5
  14. 2025Bias in Gender Bias Benchmarks: How Spurious Features Distort Evaluationcited by 5
  15. 2024Ethical-Lens: Curbing Malicious Usages of Open-Source Text-to-Image Modelscited by 4
  16. 2024Can Prompt Modifiers Control Bias? A Comparative Analysis of Text-to-Image Generative Modelscited by 4
  17. 2025Concept-as-Tree: A Controllable Synthetic Data Framework Makes Stronger Personalized VLMscited by 4
  18. 2025Deeper Diffusion Models Amplify Biascited by 4
  19. 2025Person-Centric Annotations of LAION-400M: Auditing Bias and Its Transfer to Modelscited by 4
  20. 2025How I Met Your Bias: Investigating Bias Amplification in Diffusion Modelscited by 4
  21. 2025Attention IoU: Examining Biases in CelebA using Attention Mapscited by 3
  22. 2025Model-Agnostic Gender Bias Control for Text-to-Image Generation via Sparse Autoencodercited by 3
  23. 2025Fully Unsupervised Self-debiasing of Text-to-Image Diffusion Modelscited by 3
  24. 2026Position: Don't Just "Fix it in Post": A Science of AI Must Study Training Dynamicscited by 3
  25. 2024FoolSDEdit: Deceptively Steering Your Edits Towards Targeted Attribute-aware Distributioncited by 2
  26. 2024A Picture is Worth 500 Labels: A Case Study of Demographic Disparities in Local Machine Learning Models for Instagram and TikTokcited by 2
  27. 2024GradBias: Unveiling Word Influence on Bias in Text-to-Image Generative Modelscited by 2
  28. 2025BiasConnect: Investigating Bias Interactions in Text-to-Image Modelscited by 2
  29. 2026Aesthetics as Structural Harm: Algorithmic Lookism Across Text-to-Image Generation and Classificationcited by 2
  30. 2026When Understanding Becomes a Risk: Authenticity and Safety Risks in the Emerging Image Generation Paradigmcited by 2
  31. 2026Stay Fair! Ensuring Group Fairness in Diffusion Models Across Guidance Scalescited by 2
  32. 2024Position: Towards Implicit Prompt For Text-To-Image Modelscited by 1
  33. 2024DPA: A one-stop metric to measure bias amplification in classification datasetscited by 1
  34. 2025A Woman with a Knife or A Knife with a Woman? Measuring Directional Bias Amplification in Image Captionscited by 1
  35. 2025Synthetic History: Evaluating Visual Representations of the Past in Diffusion Modelscited by 1
  36. 2025Inference Time Debiasing Concepts in Diffusion Modelscited by 1
  37. 2025Prompting Away Stereotypes? Evaluating Bias in Text-to-Image Models for Occupationscited by 1
  38. 2025Coffee: Controllable Diffusion Fine-tuningcited by 1

50 most cited of 70 All 70 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Generated Bias: Auditing Internal Bias Dynamics of Text-To-Image Generative Models
  2. 2026BAFIS: Dataset + Framework to assess occupational Bias and Human Preference in modern Text-to-image Models
  3. 2025Hidden Bias in the Machine: Stereotypes in Text-to-Image Models
  4. 2025Do Existing Testing Tools Really Uncover Gender Bias in Text-to-Image Models?
  5. 2023Word-Level Explanations for Analyzing Bias in Text-to-Image Models
  6. 2024New Job, New Gender? Measuring the Social Bias in Image Generation Models
  7. 2025Automated Evaluation of Gender Bias Across 13 Large Multimodal Models
  8. 2023Exploring Social Bias in Downstream Applications of Text-to-Image Foundation Models