# Fully Unsupervised Self-debiasing of Text-to-Image Diffusion Models

arXiv: https://arxiv.org/abs/2512.03749 · HTML: https://mlsanity.com/paper/2512.03749/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models.
- Builds on 33 of its 47 references that are in the map.
- First posted on arXiv 3 Dec 2025.

## Placed in

- Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/)

## Builds on (most cited first, 30 of 33)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022)
- [Reproducible scaling laws for contrastive language-image learning](https://mlsanity.com/paper/2212.07143/) (2022)
- [Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods](https://mlsanity.com/paper/1804.06876/) (2018)
- [MaskGAN: Towards Diverse and Interactive Facial Image Manipulation](https://mlsanity.com/paper/1907.11922/) (2019)
- [Unified Concept Editing in Diffusion Models](https://mlsanity.com/paper/2308.14761/) (2023)
- [Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLP](https://mlsanity.com/paper/2103.00453/) (2021)
- [Diffusion Models already have a Semantic Latent Space](https://mlsanity.com/paper/2210.10960/) (2022)
- [Effective Data Augmentation With Diffusion Models](https://mlsanity.com/paper/2302.07944/) (2023)
- [DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Models](https://mlsanity.com/paper/2202.04053/) (2022)
- [Fair Diffusion: Instructing Text-to-Image Generation Models on Fairness](https://mlsanity.com/paper/2302.10893/) (2023)
- [Debiasing Vision-Language Models via Biased Prompts](https://mlsanity.com/paper/2302.00070/) (2023)
- [Editing Implicit Assumptions in Text-to-Image Diffusion Models](https://mlsanity.com/paper/2303.08084/) (2023)
- [Fair Generative Modeling via Weak Supervision](https://mlsanity.com/paper/1910.12008/) (2019)
- [Finetuning Text-to-Image Diffusion Models for Fairness](https://mlsanity.com/paper/2311.07604/) (2023)
- [The Bias Amplification Paradox in Text-to-Image Generation](https://mlsanity.com/paper/2308.00755/) (2023)
- [Self-Discovering Interpretable Diffusion Latent Directions for Responsible Text-to-Image Generation](https://mlsanity.com/paper/2311.17216/) (2023)
- [ITI-GEN: Inclusive Text-to-Image Generation](https://mlsanity.com/paper/2309.05569/) (2023)
- [Concept Algebra for (Score-Based) Text-Controlled Generative Models](https://mlsanity.com/paper/2302.03693/) (2023)
- [Balancing Act: Distribution-Guided Debiasing in Diffusion Models](https://mlsanity.com/paper/2402.18206/) (2024)
- [NoiseCLR: A Contrastive Learning Approach for Unsupervised Discovery of Interpretable Directions in Diffusion Models](https://mlsanity.com/paper/2312.05390/) (2023)
- [Analyzing Bias in Diffusion-based Face Generation Models](https://mlsanity.com/paper/2305.06402/) (2023)
- [Fair Generative Models via Transfer Learning](https://mlsanity.com/paper/2212.00926/) (2022)
- [Dissecting and Mitigating Diffusion Bias via Mechanistic Interpretability](https://mlsanity.com/paper/2503.20483/) (2025)
- [Training Unbiased Diffusion Models From Biased Dataset](https://mlsanity.com/paper/2403.01189/) (2024)
- [Debiasing Text-to-Image Diffusion Models](https://mlsanity.com/paper/2402.14577/) (2024)
- [FairGen: Enhancing Fairness in Text-to-Image Diffusion Models via Self-Discovering Latent Directions](https://mlsanity.com/paper/2412.18810/) (2024)

## Built on by (most cited first, 3 of 3)

- [Training-Free Debiasing of Diffusion Models via CLIP-Guided Denoising Optimization](https://mlsanity.com/paper/2607.00817/) (2026)
- [Efficient bias mitigation in T2I diffusion models using Concept Graphs](https://mlsanity.com/paper/2607.03397/) (2026)
- [Debias Anything: Fairness with Diversity without Supervision in Diffusion Models](https://mlsanity.com/paper/2610.01815/) (2026)

## Abstract (arXiv)

Text-to-image (T2I) diffusion models have achieved widespread success due to their ability to generate high-resolution, photorealistic images. These models are trained on large-scale datasets, like LAION-5B, often scraped from the internet. However, since this data contains numerous biases, the models inherently learn and reproduce them, resulting in stereotypical outputs. We introduce SelfDebias, a fully unsupervised test-time debiasing method applicable to any diffusion model that uses a UNet as its noise predictor. SelfDebias identifies semantic clusters in an image encoder's embedding space and uses these clusters to guide the diffusion process during inference, minimizing the KL divergence between the output distribution and the uniform distribution. Unlike supervised approaches, SelfDebias does not require human-annotated datasets or external classifiers trained for each generated concept. Instead, it is designed to automatically identify semantic modes. Extensive experiments show that SelfDebias generalizes across prompts and diffusion model architectures, including both conditional and unconditional models. It not only effectively debiases images along key demographic dimensions while maintaining the visual fidelity of the generated images, but also more abstract concepts for which identifying biases is also challenging.