Safe Text-to-Image Generation: Simply Sanitize the Prompt Embedding
Not on the map. Why?
Abstract (arXiv)
In recent years, text-to-image (T2I) generation models have made significant progress in generating high-quality images that align with text descriptions. However, these models also face the risk of unsafe generation, potentially producing harmful content that violates usage policies, such as explicit material. Existing safe generation methods typically focus on suppressing inappropriate content by erasing undesired concepts from visual representations, while neglecting to sanitize the textual representation. Although these methods help mitigate the risk of misuse to some extent, their robustness remains insufficient when dealing with adversarial attacks. Given that semantic consistency between input text and output image is a core requirement of T2I models, we identify that textual representations are likely the primary source of unsafe generation. To this end, we propose Embedding Sanitizer (ES), which enhances the safety of T2I models by sanitizing inappropriate concepts in prompt embeddings. To our knowledge, ES is the first interpretable safe generation framework that assigns a score to each token in the prompt to indicate its potential harmfulness. In addition, ES adopts a plug-and-play modular design, offering compatibility for seamless integration with various T2I models and other safeguards. Evaluations on five prompt benchmarks show that ES outperforms eleven existing safeguard baselines, achieving state-of-the-art robustness while maintaining high-quality image generation.
Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 35 of its 52 references that are in the map. First posted on arXiv 15 Nov 2024.
Builds on
35 of 52 references.
References · 35
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2014Microsoft COCO: Common Objects in Contextcited by 17,498
- 2024The Llama 3 Herd of Modelscited by 14,192
- 2020Denoising Diffusion Implicit Modelscited by 9,655
- 2021Zero-Shot Text-to-Image Generationcited by 4,276
- 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
- 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
- 2020Knowledge Distillation: A Surveycited by 1,578
- 2018Datasheets for Datasetscited by 1,371
23 more
- 2019Machine Unlearningcited by 1,020
- 2023Erasing Concepts from Diffusion Modelscited by 530
- 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Modelscited by 501
- 2021Multimodal datasets: misogyny, pornography, and malignant stereotypescited by 326
- 2023Ablating Concepts in Text-to-Image Diffusion Modelscited by 312
- 2024MACE: Mass Concept Erasure in Diffusion Modelscited by 298
- 2022Easily Accessible Text-to-Image Generation Amplifies Demographic Stereotypes at Large Scalecited by 262
- 2022Red-Teaming the Stable Diffusion Safety Filtercited by 257
- 2023Ring-A-Bell! How Reliable are Concept Removal Methods for Diffusion Models?cited by 239
- 2023MMA-Diffusion: MultiModal Attack on Diffusion Modelscited by 184
- 2023SneakyPrompt: Jailbreaking Text-to-image Generative Modelscited by 175
- 2023Unsafe Diffusion: On the Generation of Unsafe Images and Hateful Memes From Text-To-Image Modelscited by 171
- 2023One-Dimensional Adapter to Rule Them All: Concepts, Diffusion Models and Erasing Applicationscited by 164
- 2022Can Machines Help Us Answering Question 16 in Datasheets, and In Turn Reflecting on Inappropriate Content?cited by 150
- 2023DiffiT: Diffusion Vision Transformers for Image Generationcited by 104
- 2023Circumventing Concept Erasure Methods For Text-to-Image Generative Modelscited by 81
- 2024Latent Guard: a Safety Framework for Text-to-image Generationcited by 68
- 2024GuardT2I: Defending Text-to-Image Models from Adversarial Promptscited by 61
- 2022Rickrolling the Artist: Injecting Backdoors into Text Encoders for Text-to-Image Synthesiscited by 59
- 2024R.A.C.E.: Robust Adversarial Concept Erasure for Secure Text-to-Image Diffusion Modelcited by 56
- 2024Universal Prompt Optimizer for Safe Text-to-Image Generationcited by 27
- 2024Moderator: Moderating Text-to-Image Diffusion Models through Fine-grained Context-based Policiescited by 15
- 2024Evaluating Semantic Variation in Text-to-Image Synthesis: A Causal Perspectivecited by 4
Built on by
Citing papers per year: 2025: 2; 2026: 2.
4 papers.
- 2025Erasing Concepts, Steering Generations: A Comprehensive Survey of Concept Suppressioncited by 10
- 2025Responsible Diffusion: A Comprehensive Survey on Safety, Ethics, and Trust in Diffusion Modelscited by 3
- 2026SafeRedir: Prompt Embedding Redirection for Robust Unlearning in Image Generation Modelscited by 3
- 2026AEGIS: A Mechanism-Guided Defense against Visual Synonym Jailbreaks in Text-to-Image Modelscited by 0
- 2026AEGIS: A Mechanism-Guided Defense against Visual Synonym Jailbreaks in Text-to-Image Models7 Jul 2026
- 2026SafeRedir: Prompt Embedding Redirection for Robust Unlearning in Image Generation Models13 Jan 2026
- 2025Responsible Diffusion: A Comprehensive Survey on Safety, Ethics, and Trust in Diffusion Models25 Sep 2025
- 2025Erasing Concepts, Steering Generations: A Comprehensive Survey of Concept Suppression26 May 2025