mlsanityData to 14 Oct 2026

GuardPaint:SpeculativeSafetyDecodingforText-to-ImageGeneration

arXiv 2608.21869 ↗First posted 22 Aug 2026cs.CV cs.AIcited by 0

On the map: Diffusion models → Robustness, privacy and adversarial aspects

also in 1 more
Abstract (arXiv)

Text-to-image (T2I) diffusion models offer powerful visual generation, but their controllability creates a critical safety challenge: adversarial prompts can steer the denoising trajectory toward policy-violating content such as explicit nudity or graphic violence. Existing safeguards mostly act before generation through prompt filtering or after generation through image classification, leaving the diffusion process itself unguarded and often yielding only refusal rather than safe visual repair. We introduce GuardPaint, a speculative decoding framework for safe T2I generation that intervenes inside the diffusion trajectory without modifying the base model. A lightweight auditor monitors intermediate images, localizes unsafe regions, and triggers surgical inpainting repair only where needed. Candidate repairs are generated by a policy-aligned inpainter and selected through a guarded tournament that accepts edits only when they improve policy compliance while preserving prompt fidelity and perceptual quality. Across five jailbreak families SneakPrompt, MMA, PGJ, DACA, and RABell and UNet/flow-matching models including SD~1.5, SDXL, SD~3.5, and FLUX.1-dev. GuardPaint reduces attack success and harmful generations with minimal degradation to image quality, prompt fidelity, and benign behavior. Content warning: This paper contains examples involving nudity and violence that some readers may find disturbing, distressing, or offensive.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models. Builds on 27 of its 44 references that are in the map. First posted on arXiv 22 Aug 2026.

Builds on

27 of 44 references.

Closest ancestors · 5

Same direction, newest first

  1. 2024Jailbreaking Prompt Attack: A Controllable Adversarial Attack against Diffusion Models2 Apr
  2. 2023MMA-Diffusion: MultiModal Attack on Diffusion Models29 Nov
  3. 2023Ring-A-Bell! How Reliable are Concept Removal Methods for Diffusion Models?16 Oct
  4. 2023SneakyPrompt: Jailbreaking Text-to-image Generative Models20 May
  5. 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Models9 Nov

Other references · 22

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  5. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
  6. 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,181
  7. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  8. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,332
  9. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
  10. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
  11. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 1,006
  12. 2021Blended Diffusion for Text-driven Editing of Natural Imagescited by 927
10 more
  1. 2024KTO: Model Alignment as Prospect Theoretic Optimizationcited by 264
  2. 2023Efficient Diffusion Training via Min-SNR Weighting Strategycited by 246
  3. 2024JailBreakV: A Benchmark for Assessing the Robustness of MultiModal Large Language Models against Jailbreak Attackscited by 230
  4. 2024Aligning Diffusion Models by Optimizing Human Utilitycited by 101
  5. 2025T2ISafety: Benchmark for Assessing Fairness, Toxicity, and Privacy in Image Generationcited by 48
  6. 2024Margin-aware Preference Optimization for Aligning Diffusion Models without Referencecited by 46
  7. 2024Binary Classifier Optimization for Large Language Model Alignmentcited by 45
  8. 2024PrefPaint: Aligning Image Inpainting Diffusion Model with Human Preferencecited by 17
  9. 2025Preference-Based Alignment of Discrete Diffusion Modelscited by 13
  10. 2025Beyond the Safety Tax: Mitigating Unsafe Text-to-Image Generation via External Safety Rectificationcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Purify Once, Edit Freely: Breaking Image Protections under Model Mismatch
  2. 2026Safe Image Generation via Reinforcement Learning
  3. 2025GuardDoor: Safeguarding Against Malicious Diffusion Editing via Protective Backdoors
  4. 2025AdvPaint: Protecting Images from Inpainting Manipulation via Adversarial Attention Disruption
  5. 2026Test-Time Scaling for Safe Text-Guided Image Generation via Intermediate Clean Estimates
  6. 2024Disrupting Diffusion-based Inpainters with Semantic Digression
  7. 2024DiffGuard: Text-Based Safety Checker for Diffusion Models
  8. 2024DiffusionGuard: A Robust Defense Against Malicious Diffusion-based Image Editing