mlsanityData to 14 Oct 2026

SafeGen: Mitigating Sexually Explicit Content Generation in Text-to-Image Models

arXiv 2404.06666 ↗First posted 10 Apr 2024cs.CV cs.AI cs.CLcited by 66

On the map: Diffusion models → Robustness, privacy and adversarial aspects

also in 1 more
Abstract (arXiv)

Text-to-image (T2I) models, such as Stable Diffusion, have exhibited remarkable performance in generating high-quality images from text descriptions in recent years. However, text-to-image models may be tricked into generating not-safe-for-work (NSFW) content, particularly in sexually explicit scenarios. Existing countermeasures mostly focus on filtering inappropriate inputs and outputs, or suppressing improper text embeddings, which can block sexually explicit content (e.g., naked) but may still be vulnerable to adversarial prompts -- inputs that appear innocent but are ill-intended. In this paper, we present SafeGen, a framework to mitigate sexual content generation by text-to-image models in a text-agnostic manner. The key idea is to eliminate explicit visual representations from the model regardless of the text input. In this way, the text-to-image model is resistant to adversarial prompts since such unsafe visual representations are obstructed from within. Extensive experiments conducted on four datasets and large-scale user studies demonstrate SafeGen's effectiveness in mitigating sexually explicit content generation while preserving the high-fidelity of benign images. SafeGen outperforms eight state-of-the-art baseline methods and achieves 99.4% sexual content removal performance. Furthermore, our constructed benchmark of adversarial prompts provides a basis for future development and evaluation of anti-NSFW-generation methods.

Key facts. Cited by 66 papers in the map (data through 14 Oct 2026). Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models. Builds on 27 of its 52 references that are in the map. First posted on arXiv 10 Apr 2024.

Builds on

27 of 52 references.

Closest ancestors · 7

Same direction, newest first

  1. 2023Ring-A-Bell! How Reliable are Concept Removal Methods for Diffusion Models?16 Oct
  2. 2023Evaluating the Robustness of Text-to-image Diffusion Models against Real-world Attacks16 Jun
  3. 2023Unsafe Diffusion: On the Generation of Unsafe Images and Hateful Memes From Text-To-Image Models23 May
  4. 2023SneakyPrompt: Jailbreaking Text-to-image Generative Models20 May
  5. 2023Erasing Concepts from Diffusion Models13 Mar
  6. 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Models9 Nov
  7. 2022Red-Teaming the Stable Diffusion Safety Filter3 Oct

Other references · 20

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  4. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  6. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  7. 2020Denoising Diffusion Implicit Modelscited by 9,781
  8. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  9. 2022Classifier-Free Diffusion Guidancecited by 5,847
  10. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  11. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,614
  12. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,778
8 more
  1. 2020DiffWave: A Versatile Diffusion Model for Audio Synthesiscited by 1,354
  2. 2022Denoising Diffusion Restoration Modelscited by 963
  3. 2022DiffuSeq: Sequence to Sequence Text Generation with Diffusion Modelscited by 450
  4. 2023Circumventing Concept Erasure Methods For Text-to-Image Generative Modelscited by 85
  5. 2021U-Net Transformer: Self and Cross Attention for Medical Image Segmentationcited by 81
  6. 2020Normalized and Geometry-Aware Self-Attention Network for Image Captioningcited by 63
  7. 2023Typology of Risks of Generative Text-to-Image Modelscited by 55
  8. 2023Mitigating Inappropriateness in Image Generation: Can there be Value in Reflecting the World's Ugliness?cited by 16

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 10; 2025: 33; 2026: 23.

66 papers.

Sort:

By area:

  1. 2024Direct Unlearning Optimization for Robust and Safe Text-to-Image Modelscited by 50
  2. 2024VBench++: Comprehensive and Versatile Benchmark Suite for Video Generative Modelscited by 45
  3. 2024Heuristic-Induced Multimodal Risk Distribution Jailbreak Attack for Multimodal Large Language Modelscited by 33
  4. 2024Fuzz-Testing Meets LLM-Based Agents: An Automated and Efficient Framework for Jailbreaking Text-To-Image Generation Modelscited by 27
  5. 2025Rethinking Data Protection in the (Generative) Artificial Intelligence Eracited by 18
  6. 2025PromptGuard: Soft Prompt-Guided Unsafe Content Moderation for Text-to-Image Modelscited by 17
  7. 2025A Vision for Access Control in LLM-based Agent Systemscited by 14
  8. 2024Replication in Visual Diffusion Models: A Survey and Outlookcited by 11
  9. 2025CHAI: Command Hijacking against embodied AIcited by 11
  10. 2024Towards Understanding Unsafe Video Generationcited by 10
  11. 2024DiffVax: Optimization-Free Image Immunization Against Diffusion-Based Editingcited by 10
  12. 2025Erased but Not Forgotten: How Backdoors Compromise Concept Erasurecited by 9
38 more
  1. 2025Towards Resilient Safety-driven Unlearning for Diffusion Models against Downstream Fine-tuningcited by 9
  2. 2025PLA: Prompt Learning Attack against Text-to-Image Generative Modelscited by 9
  3. 2025Sparse Autoencoder as a Zero-Shot Classifier for Concept Erasing in Text-to-Image Diffusion Modelscited by 8
  4. 2025Red-Teaming Text-to-Image Systems by Rule-based Preference Modelingcited by 8
  5. 2024ShieldDiff: Suppressing Sexual Content Generation from Diffusion Models through Reinforcement Learningcited by 7
  6. 2024Safeguarding Text-to-Image Generation via Inference-Time Prompt-Noise Optimizationcited by 7
  7. 2025Mitigating Sexual Content Generation via Embedding Distortion in Text-conditioned Diffusion Modelscited by 7
  8. 2025Concept Steerers: Leveraging K-Sparse Autoencoders for Test-Time Controllable Generationscited by 7
  9. 2025SafeText: Safe Text-to-image Models via Aligning the Text Encodercited by 7
  10. 2026GRP-Obliteration: Unaligning LLMs With a Single Unlabeled Promptcited by 7
  11. 2025Evolution-based Region Adversarial Prompt Learning for Robustness Enhancement in Vision-Language Modelscited by 6
  12. 2025DyME: Dynamic Multi-Concept Erasure in Diffusion Models with Bi-Level Orthogonal LoRA Adaptationcited by 6
  13. 2025SafeGuider: Robust and Practical Content Safety Control for Text-to-Image Modelscited by 6
  14. 2025Dynamic Optimization and Safety Indicator Injection for Jailbreaking Text-to-Image Models with Multimodal Safety Filterscited by 5
  15. 2025DREAM: Scalable Red Teaming for Text-to-Image Generative Systems via Distribution Modelingcited by 5
  16. 2025Generative Model Unlearning: A Survey through Target Events, Unlearning Operators, and Evaluation Protocolscited by 4
  17. 2025SafeVision: Efficient Image Guardrail with Robust Policy Adherence and Explainabilitycited by 4
  18. 2026ActErase: A Training-Free Paradigm for Precise Concept Erasure via Activation Redirectioncited by 4
  19. 2024Buster: Implanting Semantic Backdoor into Text Encoder to Mitigate NSFW Content Generationcited by 3
  20. 2026Conditioned Activation Transport for T2I Safety Steeringcited by 3
  21. 2025A Systematic Review of Open Datasets Used in Text-to-Image (T2I) Gen AI Model Safetycited by 2
  22. 2025Mind the Trojan Horse: Image Prompt Adapter Enabling Scalable and Deceptive Jailbreakingcited by 2
  23. 2025REDEditing: Relationship-Driven Precise Backdoor Poisoning on Text-to-Image Diffusion Modelscited by 2
  24. 2025PromptSafe: Gated Prompt Tuning for Safe Text-to-Image Generationcited by 2
  25. 2025Beyond the Safety Tax: Mitigating Unsafe Text-to-Image Generation via External Safety Rectificationcited by 2
  26. 2025SafetyPairs: Isolating Safety Critical Image Features with Counterfactual Image Generationcited by 2
  27. 2026When Understanding Becomes a Risk: Authenticity and Safety Risks in the Emerging Image Generation Paradigmcited by 2
  28. 2026FlowErase-RL: Rethinking Concept Erasure as Reward Optimization in Flow Matching Modelscited by 2
  29. 2025Do Concept Replacement Techniques Really Erase Unacceptable Concepts?cited by 1
  30. 2025LoRAShield: Data-Free Editing Alignment for Secure Personalized LoRA Sharingcited by 1
  31. 2025Patronus: Safeguarding Text-to-Image Models against White-Box Adversariescited by 1
  32. 2025Towards Irreversible Machine Unlearning for Diffusion Modelscited by 1
  33. 2026The Illusion of Forgetting: Attack Unlearned Diffusion via Initial Latent Variable Optimizationcited by 1
  34. 2026DTVI: Dual-Stage Textual and Visual Intervention for Safe Text-to-Image Generationcited by 1
  35. 2026Harnessing Hyperbolic Geometry for Harmful Prompt Detection and Sanitizationcited by 1
  36. 2026Erased but Exploitable: Black-box Embedding-Aware Prompting Against Unlearned Text-to-Image Diffusion Modelscited by 1
  37. 2026Pulling The REINS: Training-Free Safety Alignment of Video Diffusion Models via Representation Steeringcited by 1
  38. 2026The Illusion of High Utility in Safety Alignment of Text-to-Image Diffusion Modelscited by 1

50 most cited of 66 All 66 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2024Safe Text-to-Image Generation: Simply Sanitize the Prompt Embedding
  2. 2025NDM: A Noise-driven Detection and Mitigation Framework against Implicit Sexual Intentions in Text-to-Image Generation
  3. 2026UniNDM: A Unified Noise-driven Detection and Mitigation Framework Against Sexual Content in Text-to-Image Generation
  4. 2026Harmful Content Generation in Text-to-Image Models: Capabilities and Moderation Limitations
  5. 2025Training-Free Safe Text Embedding Guidance for Text-to-Image Diffusion Models
  6. 2023Safe-CLIP: Removing NSFW Concepts from Vision-and-Language Models
  7. 2026SafeGen-Bench: Benchmarking Safety in Image-Conditioned Text-to-Video Generation
  8. 2024When Image Generation Goes Wrong: A Safety Analysis of Stable Diffusion Models