# Model-Agnostic Gender Bias Control for Text-to-Image Generation via Sparse Autoencoder arXiv: https://arxiv.org/abs/2507.20973 ยท HTML: https://mlsanity.com/paper/2507.20973/ ## Key facts - Cited by 3 papers in the map (data through 14 Oct 2026). - Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. - Builds on 34 of its 44 references that are in the map. - First posted on arXiv 28 Jul 2025. ## Placed in - Diffusion models / [Image generation, text-to-image and editing](https://mlsanity.com/diffusion/image-generation-text-to-image-and-editing/) - Mechanistic interpretability / [Interpreting and steering diffusion models](https://mlsanity.com/mechinterp/interpreting-and-steering-diffusion-models/) ## Builds on (most cited first, 30 of 34) - [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021) - [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021) - [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022) - [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023) - [LAION-5B: An open large-scale dataset for training next generation image-text models](https://mlsanity.com/paper/2210.08402/) (2022) - [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016) - [CLIPScore: A Reference-free Evaluation Metric for Image Captioning](https://mlsanity.com/paper/2104.08718/) (2021) - [Sparse Autoencoders Find Highly Interpretable Features in Language Models](https://mlsanity.com/paper/2309.08600/) (2023) - [Scaling and evaluating sparse autoencoders](https://mlsanity.com/paper/2406.04093/) (2024) - [Unified Concept Editing in Diffusion Models](https://mlsanity.com/paper/2308.14761/) (2023) - [Bias in Bios: A Case Study of Semantic Representation Bias in a High-Stakes Setting](https://mlsanity.com/paper/1901.09451/) (2019) - [What the DAAM: Interpreting Stable Diffusion Using Cross Attention](https://mlsanity.com/paper/2210.04885/) (2022) - [DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Models](https://mlsanity.com/paper/2202.04053/) (2022) - [Stable Bias: Analyzing Societal Representations in Diffusion Models](https://mlsanity.com/paper/2303.11408/) (2023) - [k-Sparse Autoencoders](https://mlsanity.com/paper/1312.5663/) (2013) - [Fair Diffusion: Instructing Text-to-Image Generation Models on Fairness](https://mlsanity.com/paper/2302.10893/) (2023) - [Debiasing Vision-Language Models via Biased Prompts](https://mlsanity.com/paper/2302.00070/) (2023) - [Editing Implicit Assumptions in Text-to-Image Diffusion Models](https://mlsanity.com/paper/2303.08084/) (2023) - [Are Gender-Neutral Queries Really Gender-Neutral? Mitigating Gender Bias in Image Search](https://mlsanity.com/paper/2109.05433/) (2021) - [Sparse Autoencoders Learn Monosemantic Features in Vision-Language Models](https://mlsanity.com/paper/2504.02821/) (2025) - [DeAR: Debiasing Vision-Language Models with Additive Residuals](https://mlsanity.com/paper/2303.10431/) (2023) - [The Bias Amplification Paradox in Text-to-Image Generation](https://mlsanity.com/paper/2308.00755/) (2023) - [FLUX-Text: A Simple and Advanced Diffusion Transformer Baseline for Scene Text Editing](https://mlsanity.com/paper/2505.03329/) (2025) - [Evidence for Hypodescent in Visual Semantic AI](https://mlsanity.com/paper/2205.10764/) (2022) - [Fair Generative Models via Transfer Learning](https://mlsanity.com/paper/2212.00926/) (2022) - [CoDi: Conditional Diffusion Distillation for Higher-Fidelity and Faster Image Generation](https://mlsanity.com/paper/2310.01407/) (2023) - [Stable Diffusion Exposed: Gender Bias from Prompt to Image](https://mlsanity.com/paper/2312.03027/) (2023) - [Mitigating stereotypical biases in text to image generative systems](https://mlsanity.com/paper/2310.06904/) (2023) - [A Unified Debiasing Approach for Vision-Language Models across Modalities and Tasks](https://mlsanity.com/paper/2410.07593/) (2024) - [SCALAR: Scale-wise Controllable Visual Autoregressive Learning](https://mlsanity.com/paper/2507.19946/) (2025) ## Built on by (most cited first, 3 of 3) - [Interpretable Debiasing of Vision-Language Models for Social Fairness](https://mlsanity.com/paper/2602.24014/) (2026) - [Training-Free Debiasing of Diffusion Models via CLIP-Guided Denoising Optimization](https://mlsanity.com/paper/2607.00817/) (2026) - [Volumetric Directional Diffusion: Anchoring Uncertainty Quantification in Anatomical Consensus for Ambiguous Medical Image Segmentation](https://mlsanity.com/paper/2603.04024/) (2026) ## Abstract (arXiv) Text-to-image (T2I) diffusion models often exhibit gender bias, particularly by generating stereotypical associations between professions and gendered subjects. This paper presents SAE Debias, a lightweight and model-agnostic framework for mitigating such bias in T2I generation. Unlike prior approaches that rely on CLIP-based filtering or prompt engineering, which often require model-specific adjustments and offer limited control, SAE Debias operates directly within the feature space without retraining or architectural modifications. By leveraging a k-sparse autoencoder pre-trained on a gender bias dataset, the method identifies gender-relevant directions within the sparse latent space, capturing professional stereotypes. Specifically, a biased direction per profession is constructed from sparse latents and suppressed during inference to steer generations toward more gender-balanced outputs. Trained only once, the sparse autoencoder provides a reusable debiasing direction, offering effective control and interpretable insight into biased subspaces. Extensive evaluations across multiple T2I models, including Stable Diffusion 1.4, 1.5, 2.1, and SDXL, demonstrate that SAE Debias substantially reduces gender bias while preserving generation quality. To the best of our knowledge, this is the first work to apply sparse autoencoders for identifying and intervening in gender bias within T2I models. These findings contribute toward building socially responsible generative AI, providing an interpretable and model-agnostic tool to support fairness in text-to-image generation.