mlsanityData to 14 Oct 2026

Mitigating stereotypical biases in text to image generative systems

arXiv 2310.06904 ↗First posted 10 Oct 2023cs.CVcited by 19

Not on the map. Why?

Abstract (arXiv)

State-of-the-art generative text-to-image models are known to exhibit social biases and over-represent certain groups like people of perceived lighter skin tones and men in their outcomes. In this work, we propose a method to mitigate such biases and ensure that the outcomes are fair across different groups of people. We do this by finetuning text-to-image models on synthetic data that varies in perceived skin tones and genders constructed from diverse text prompts. These text prompts are constructed from multiplicative combinations of ethnicities, genders, professions, age groups, and so on, resulting in diverse synthetic data. Our diversity finetuned (DFT) model improves the group fairness metric by 150% for perceived skin tone and 97.7% for perceived gender. Compared to baselines, DFT models generate more people with perceived darker skin tone and more women. To foster open research, we will release all text prompts and code to generate training images.

Key facts. Cited by 19 papers in the map (data through 14 Oct 2026). Builds on 26 of its 34 references that are in the map. First posted on arXiv 10 Oct 2023.

Builds on

26 of 34 references.

References · 26

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,608
  4. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  5. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  6. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  7. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  8. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  9. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  10. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
  11. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,476
  12. 2020StereoSet: Measuring stereotypical bias in pretrained language modelscited by 924
14 more
  1. 2014Certifying and removing disparate impactcited by 715
  2. 2023Text2Video-Zero: Text-to-Image Diffusion Models are Zero-Shot Video Generatorscited by 670
  3. 2023Structure and Content-Guided Video Synthesis with Diffusion Modelscited by 590
  4. 2017Men Also Like Shopping: Reducing Gender Bias Amplification using Corpus-level Constraintscited by 420
  5. 2023SVDiff: Compact Parameter Space for Diffusion Fine-Tuningcited by 338
  6. 2023Pix2Video: Video Editing using Image Diffusioncited by 320
  7. 2023Emu: Enhancing Image Generation Models Using Photogenic Needles in a Haystackcited by 279
  8. 2018Women also Snowboard: Overcoming Bias in Captioning Modelscited by 236
  9. 2023Stable Bias: Analyzing Societal Representations in Diffusion Modelscited by 226
  10. 2023Fair Diffusion: Instructing Text-to-Image Generation Models on Fairnesscited by 151
  11. 2021Understanding and Evaluating Racial Biases in Image Captioningcited by 108
  12. 2020Image Representations Learned With Unsupervised Pre-Training Contain Human-like Biasescited by 107
  13. 2022How well can Text-to-Image Generative Models understand Ethical Natural Language Interventions?cited by 92
  14. 2023Auditing Gender Presentation Differences in Text-to-Image Modelscited by 22

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 7; 2025: 8; 2026: 3.

19 papers.

Sort:

By area:

  1. 2024Large-scale Reinforcement Learning for Diffusion Modelscited by 78
  2. 2024Survey of Bias In Text-to-Image Generation: Definition, Evaluation, and Mitigationcited by 66
  3. 2023TIBET: Identifying and Evaluating Biases in Text-to-Image Generative Modelscited by 52
  4. 2025A Large Scale Analysis of Gender Biases in Text-to-Image Generative Modelscited by 14
  5. 2024Fairness and Bias Mitigation in Computer Vision: A Surveycited by 13
  6. 2025Concept Steerers: Leveraging K-Sparse Autoencoders for Test-Time Controllable Generationscited by 7
  7. 2025Increasing the Utility of Synthetic Images through Chamfer Guidancecited by 6
  8. 2025Do Existing Testing Tools Really Uncover Gender Bias in Text-to-Image Models?cited by 4
  9. 2024They're All Doctors: Synthesizing Diverse Counterfactuals to Mitigate Associative Biascited by 3
  10. 2025Model-Agnostic Gender Bias Control for Text-to-Image Generation via Sparse Autoencodercited by 3
  11. 2024Embedding an Ethical Mind: Aligning Text-to-Image Synthesis via Lightweight Value Optimizationcited by 2
  12. 2025BiasConnect: Investigating Bias Interactions in Text-to-Image Modelscited by 2
7 more
  1. 2025Mitigate One, Skew Another? Tackling Intersectional Biases in Text-to-Image Modelscited by 2
  2. 2024AITTI: Learning Adaptive Inclusive Token for Text-to-Image Generationcited by 1
  3. 2024MoESD: Mixture of Experts Stable Diffusion to Mitigate Gender Biascited by 1
  4. 2025SustainDiffusion: Optimising the Social and Environmental Sustainability of Stable Diffusion Modelscited by 0
  5. 2026SoS: Analysis of Surface over Semantics in Multilingual Text-To-Image Generationcited by 0
  6. 2026Investigating Disability Representations in Text-to-Image Modelscited by 0
  7. 2026KG-FairDiff: Knowledge Graph-Guided Prompt Refinement for Demographically Fair Text-to-Image Generationcited by 0