mlsanityData to 14 Oct 2026

Fair Text-to-Image Diffusion via Fair Mapping

arXiv 2311.17695 ↗First posted 29 Nov 2023cs.CV cs.AI cs.CYcited by 31

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

In this paper, we address the limitations of existing text-to-image diffusion models in generating demographically fair results when given human-related descriptions. These models often struggle to disentangle the target language context from sociocultural biases, resulting in biased image generation. To overcome this challenge, we propose Fair Mapping, a flexible, model-agnostic, and lightweight approach that modifies a pre-trained text-to-image diffusion model by controlling the prompt to achieve fair image generation. One key advantage of our approach is its high efficiency. It only requires updating an additional linear network with few parameters at a low computational cost. By developing a linear network that maps conditioning embeddings into a debiased space, we enable the generation of relatively balanced demographic results based on the specified text condition. With comprehensive experiments on face image generation, we show that our method significantly improves image generation fairness with almost the same image quality compared to conventional diffusion models when prompted with descriptions related to humans. By effectively addressing the issue of implicit language bias, our method produces more fair and diverse image outputs.

Key facts. Cited by 31 papers in the map (data through 14 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 51 of its 66 references that are in the map. First posted on arXiv 29 Nov 2023.

Builds on

51 of 66 references.

Closest ancestors · 10

Same direction, newest first

  1. 2023Finetuning Text-to-Image Diffusion Models for Fairness11 Nov
  2. 2023Dense Text-to-Image Generation with Attention Modulation24 Aug
  3. 2023Editing Implicit Assumptions in Text-to-Image Diffusion Models14 Mar
  4. 2023Text-to-image Diffusion Models in Generative AI: A Survey14 Mar
  5. 2023Uncovering Bias in Face Generation Models22 Feb
  6. 2022Training-Free Structured Diffusion Guidance for Compositional Text-to-Image Synthesis9 Dec
  7. 2022SpaText: Spatio-Textual Representation for Controllable Image Generation25 Nov
  8. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers2 Nov
  9. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding23 May
  10. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr

Other references · 41

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  5. 2020Denoising Diffusion Implicit Modelscited by 9,781
  6. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  7. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  8. 2022Classifier-Free Diffusion Guidancecited by 5,847
  9. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  10. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  11. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  12. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
29 more
  1. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,424
  2. 2016Equality of Opportunity in Supervised Learningcited by 2,007
  3. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,555
  4. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,476
  5. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
  6. 2022Human Motion Diffusion Modelcited by 1,080
  7. 2022Mass-Editing Memory in a Transformercited by 1,001
  8. 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Modelscited by 513
  9. 2023Unified Concept Editing in Diffusion Modelscited by 388
  10. 2021Persistent Anti-Muslim Bias in Large Language Modelscited by 377
  11. 2021Multimodal datasets: misogyny, pornography, and malignant stereotypescited by 327
  12. 2023Pix2Video: Video Editing using Image Diffusioncited by 320
  13. 2023Any-to-Any Generation via Composable Diffusioncited by 204
  14. 2018FairGAN: Fairness-aware Generative Adversarial Networkscited by 164
  15. 2023Fair Diffusion: Instructing Text-to-Image Generation Models on Fairnesscited by 151
  16. 2023Debiasing Vision-Language Models via Biased Promptscited by 149
  17. 2022A Prompt Array Keeps the Bias Away: Debiasing Vision-Language Models with Adversarial Learningcited by 104
  18. 2023Elucidating the Exposure Bias in Diffusion Modelscited by 101
  19. 2022How well can Text-to-Image Generative Models understand Ethical Natural Language Interventions?cited by 92
  20. 2022Membership Inference Attacks Against Text-to-image Generation Modelscited by 69
  21. 2023Toward Verifiable and Reproducible Human Evaluation for Text-to-Image Generationcited by 68
  22. 2023ReFACT: Updating Text-to-Image Models by Editing the Text Encodercited by 32
  23. 2022Counterfactually Measuring and Eliminating Social Bias in Vision-Language Pre-training Modelscited by 28
  24. 2023Localized Text-to-Image Generation for Free via Cross Attention Controlcited by 27
  25. 2023A Friendly Face: Do Text-to-Image Systems Rely on Stereotypes when the Input is Under-Specified?cited by 25
  26. 2022FairCLIP: Social Bias Elimination based on Attribute Prototype Learning and Representation Neutralizationcited by 20
  27. 2021Word Embeddings via Causal Inference: Gender Bias Reducing and Semantic Information Preservingcited by 11
  28. 2018Polarity and Intensity: the Two Aspects of Sentiment Analysiscited by 8
  29. 2022MixFairFace: Towards Ultimate Fairness via MixFair Adapter in Face Recognitioncited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 8; 2025: 11; 2026: 12.

31 papers.

Sort:

By area:

  1. 2024Survey of Bias In Text-to-Image Generation: Definition, Evaluation, and Mitigationcited by 66
  2. 2024SATO: Stable Text-to-Motion Frameworkcited by 20
  3. 2024Text Guided Image Editing with Automatic Concept Locating and Forgettingcited by 12
  4. 2024Tackling Noisy Clients in Federated Learning with End-to-end Label Correctioncited by 12
  5. 2024Investigating Large Language Models for Code Vulnerability Detection: An Experimental Studycited by 12
  6. 2025FairImagen: Post-Processing for Bias Mitigation in Text-to-Image Modelscited by 7
  7. 2024Trustworthy Text-to-Image Diffusion Models: A Timely and Focused Surveycited by 6
  8. 2025FNBench: Benchmarking Robust Federated Learning against Noisy Labelscited by 6
  9. 2024FairSkin: Fair Diffusion for Skin Disease Image Generationcited by 4
  10. 2025TransDiffuser: Diverse Trajectory Generation with Decorrelated Multi-modal Representation for End-to-end Autonomous Drivingcited by 4
  11. 2026TP-Blend: Textual-Prompt Attention Pairing for Precise Object-Style Blending in Diffusion Modelscited by 4
  12. 2025Multi-Group Proportional Representation for Text-to-Image Modelscited by 3
19 more
  1. 2025Beyond Overcorrection: Evaluating Diversity in T2I Models with DivBenchcited by 2
  2. 2025FAME: Fairness-aware Attention-modulated Video Editingcited by 2
  3. 2026Inference-Time Attribute Distribution Alignment for Unconditional Diffusioncited by 2
  4. 2026Stay Fair! Ensuring Group Fairness in Diffusion Models Across Guidance Scalescited by 2
  5. 2026Multi-Axis Max@K Reinforcement Learning for Representative Diversity in Text-to-Image Generationcited by 2
  6. 2024AITTI: Learning Adaptive Inclusive Token for Text-to-Image Generationcited by 1
  7. 2025What Is the Point of Equality in Machine Learning Fairness? Beyond Equality of Opportunitycited by 1
  8. 2025Prompting Away Stereotypes? Evaluating Bias in Text-to-Image Models for Occupationscited by 1
  9. 2025ZoFia: Zero-Shot Fake News Detection with Entity-Guided Retrieval and Multi-LLM Interactioncited by 1
  10. 2026HoloFair: Unified T2I Fairness Evaluation and Fair-GRPO Debiasingcited by 1
  11. 2026Training-Free Debiasing of Diffusion Models via CLIP-Guided Denoising Optimizationcited by 1
  12. 2025A Framework for Benchmarking Fairness-Utility Trade-offs in Text-to-Image Models via Pareto Frontierscited by 0
  13. 2025FairJudge: Abstention-Aware Multimodal Judges for Fairness and Alignment Evaluation in Text-to-Image Modelscited by 0
  14. 2026Operationalizing Fairness in Text-to-Image Models: A Survey of Bias, Fairness Audits and Mitigation Strategiescited by 0
  15. 2026Orthogonal Negative Guidance in Attention Feature Space for Text-to-Image Generationcited by 0
  16. 2026KG-FairDiff: Knowledge Graph-Guided Prompt Refinement for Demographically Fair Text-to-Image Generationcited by 0
  17. 2026Intermediate Text Representation Guided Text-to-Image Generation for Enhancing One-and-Only Alignmentcited by 0
  18. 2026FairFlow: Demystifying and Mitigating Stereotype Bias in Text-to-Image Diffusion Transformerscited by 0
  19. 2026Efficient bias mitigation in T2I diffusion models using Concept Graphscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025BiasMap: Leveraging Cross-Attentions to Discover and Mitigate Hidden Social Biases in Text-to-Image Generation
  2. 2026Debias Anything: Fairness with Diversity without Supervision in Diffusion Models
  3. 2024Bias Begets Bias: The Impact of Biased Embeddings on Diffusion Models
  4. 2024FairRAG: Fair Human Generation via Fair Retrieval Augmentation
  5. 2024Rethinking Training for De-biasing Text-to-Image Generation: Unlocking the Potential of Stable Diffusion
  6. 2026T2I-BiasBench: A Multi-Metric Framework for Auditing Demographic and Cultural Bias in Text-to-Image Models
  7. 2024FairGen: Enhancing Fairness in Text-to-Image Diffusion Models via Self-Discovering Latent Directions
  8. 2023Diverse Diffusion: Enhancing Image Diversity in Text-to-Image Generation