Safe Image Generation via Reinforcement Learning
On the map: Deep reinforcement learning methods → RL with diffusion and flow models
Abstract (arXiv)
Recent Text-to-Image (T2I) models achieve remarkable visual image generation performance, but they can still generate NSFW (Not-Safe-For-Work) contents, including violent or explicit images. Existing safety checker mechanisms are largely confined to pre-generation filtering (e.g. prompt-level text classifiers) or post-hoc moderation applied after an image is completely synthesized. However, adversarial attack methods operate over a much broader space. This imbalance highlights the need for a safety mechanism that intervenes during the generation process. We propose an in-generation safety framework that monitors the denoising trajectory and detects emerging NSFW signals from intermediate representations. Rather than merely detecting NSFW generations, our method applies reinforcement learning to generate safe images from NSFW prompts. By coupling in-generation detection with controllable steering, our approach mitigates unsafe trajectories even when NSFW signals emerge after generation has already begun. Experiments results show that our method consistently outperforms existing safe image generation methods across both standard and adversarial evaluation sets, while preserving perceptual quality and prompt fidelity. Code will be released upon acceptance.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL with diffusion and flow models, one of 15 directions in Deep reinforcement learning methods. Builds on 39 of its 47 references that are in the map. First posted on arXiv 5 Oct 2026.
Builds on
39 of 47 references.
References · 39
- 2020Denoising Diffusion Probabilistic Modelscited by 21,076
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2014Microsoft COCO: Common Objects in Contextcited by 17,608
- 2017Proximal Policy Optimization Algorithmscited by 15,215
- 2015Distilling the Knowledge in a Neural Networkcited by 10,324
- 2020Denoising Diffusion Implicit Modelscited by 9,781
- 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,415
- 2022Classifier-Free Diffusion Guidancecited by 5,847
- 2022Flow Matching for Generative Modelingcited by 5,702
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
- 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
- 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 2,010
27 more
- 2023Training Diffusion Models with Reinforcement Learningcited by 982
- 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Modelscited by 513
- 2015A Neural Algorithm of Artistic Stylecited by 483
- 2023DPOK: Reinforcement Learning for Fine-tuning Text-to-Image Diffusion Modelscited by 450
- 2023Directly Fine-Tuning Diffusion Models on Differentiable Rewardscited by 448
- 2023Unified Concept Editing in Diffusion Modelscited by 388
- 2022A Holistic Approach to Undesired Content Detection in the Real Worldcited by 356
- 2023Aligning Text-to-Image Diffusion Models with Reward Backpropagationcited by 288
- 2022Red-Teaming the Stable Diffusion Safety Filtercited by 264
- 2023Ring-A-Bell! How Reliable are Concept Removal Methods for Diffusion Models?cited by 250
- 2023MMA-Diffusion: MultiModal Attack on Diffusion Modelscited by 194
- 2023SneakyPrompt: Jailbreaking Text-to-image Generative Modelscited by 189
- 2023Unsafe Diffusion: On the Generation of Unsafe Images and Hateful Memes From Text-To-Image Modelscited by 175
- 2023Prompting4Debugging: Red-Teaming Text-to-Image Diffusion Models by Finding Problematic Promptscited by 171
- 2022Can Machines Help Us Answering Question 16 in Datasheets, and In Turn Reflecting on Inappropriate Content?cited by 155
- 2024Reliable and Efficient Concept Erasure of Text-to-Image Diffusion Modelscited by 133
- 2024SAFREE: Training-Free and Adaptive Guard for Safe Text-to-Image And Video Generationcited by 133
- 2024Jailbreaking Prompt Attack: A Controllable Adversarial Attack against Diffusion Modelscited by 72
- 2024Latent Guard: a Safety Framework for Text-to-image Generationcited by 71
- 2024SafeGen: Mitigating Sexually Explicit Content Generation in Text-to-Image Modelscited by 66
- 2024GuardT2I: Defending Text-to-Image Models from Adversarial Promptscited by 62
- 2024Perception-guided Jailbreak against Text-to-Image Modelscited by 40
- 2025Towards Better Alignment: Training Diffusion Models with Reinforcement Learning Against Sparse Rewardscited by 32
- 2023DiffGuard: Semantic Mismatch-Guided Out-of-Distribution Detection using Pre-trained Diffusion Modelscited by 26
- 2024DiffZOO: A Purely Query-Based Black-Box Attack for Red-teaming Text-to-Image Generative Model via Zeroth Order Optimizationcited by 22
- 2025TRAP: Targeted Redirecting of Agentic Preferencescited by 5
- 2025PurifyGen: A Risk-Discrimination and Semantic-Purification Model for Safe Text-to-Image Generationcited by 5
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2025GenBreak: Red Teaming Text-to-Image Generators Using Large Language Models
- 2023Distilling Adversarial Prompts from Safety Benchmarks: Report for the Adversarial Nibbler Challenge
- 2024Safeguarding Text-to-Image Generation via Inference-Time Prompt-Noise Optimization
- 2026Test-Time Scaling for Safe Text-Guided Image Generation via Intermediate Clean Estimates
- 2025SC-Pro: Training-Free Framework for Defending Unsafe Image Synthesis Attack
- 2026DTVI: Dual-Stage Textual and Visual Intervention for Safe Text-to-Image Generation
- 2023Adversarial Nibbler: A Data-Centric Challenge for Improving the Safety of Text-to-Image Models
- 2024ShieldDiff: Suppressing Sexual Content Generation from Diffusion Models through Reinforcement Learning