mlsanityData to 7 Oct 2026

GuardT2I: Defending Text-to-Image Models from Adversarial Prompts

arXiv 2403.01446 ↗First posted 3 Mar 2024cs.CVcited by 61

On the map: LLM safety and security → Vision-language and text-to-image safety

Abstract (arXiv)

Recent advancements in Text-to-Image (T2I) models have raised significant safety concerns about their potential misuse for generating inappropriate or Not-Safe-For-Work (NSFW) contents, despite existing countermeasures such as NSFW classifiers or model fine-tuning for inappropriate concept removal. Addressing this challenge, our study unveils GuardT2I, a novel moderation framework that adopts a generative approach to enhance T2I models' robustness against adversarial prompts. Instead of making a binary classification, GuardT2I utilizes a Large Language Model (LLM) to conditionally transform text guidance embeddings within the T2I models into natural language for effective adversarial prompt detection, without compromising the models' inherent performance. Our extensive experiments reveal that GuardT2I outperforms leading commercial solutions like OpenAI-Moderation and Microsoft Azure Moderator by a significant margin across diverse adversarial scenarios. Our framework is available at https://github.com/cure-lab/GuardT2I.

Key facts. Cited by 61 papers in the map (data through 7 Oct 2026). Placed in Vision-language and text-to-image safety, one of 12 directions in LLM safety and security. Builds on 42 of its 61 references that are in the map. First posted on arXiv 3 Mar 2024.

Builds on

42 of 61 references.

Closest ancestors · 5

Same direction, newest first

  1. 2023To Generate or Not? Safety-Driven Unlearned Diffusion Models Are Still Easy To Generate Unsafe Images ... For Now18 Oct
  2. 2023Ring-A-Bell! How Reliable are Concept Removal Methods for Diffusion Models?16 Oct
  3. 2023Prompting4Debugging: Red-Teaming Text-to-Image Diffusion Models by Finding Problematic Prompts12 Sep
  4. 2023SneakyPrompt: Jailbreaking Text-to-image Generative Models20 May
  5. 2022Red-Teaming the Stable Diffusion Safety Filter3 Oct

Other references · 37

  1. 2017Attention Is All You Needcited by 59,932
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,049
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  7. 2020Scaling Laws for Neural Language Modelscited by 6,411
  8. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  9. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,530
  10. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  11. 2019DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lightercited by 4,243
  12. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
25 more
  1. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  2. 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,286
  3. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  4. 2021BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Modelscited by 1,225
  5. 2021GLM: General Language Model Pretraining with Autoregressive Blank Infillingcited by 1,026
  6. 2019On the Convergence of Adam and Beyondcited by 834
  7. 2020Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillationcited by 591
  8. 2020A Survey on Contrastive Self-supervised Learningcited by 553
  9. 2023Erasing Concepts from Diffusion Modelscited by 530
  10. 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Modelscited by 501
  11. 2017Supervised Learning of Universal Sentence Representations from Natural Language Inference Datacited by 428
  12. 2022A Holistic Approach to Undesired Content Detection in the Real Worldcited by 337
  13. 2023Ablating Concepts in Text-to-Image Diffusion Modelscited by 312
  14. 2017Beam Search Strategies for Neural Machine Translationcited by 232
  15. 2023MMA-Diffusion: MultiModal Attack on Diffusion Modelscited by 184
  16. 2019Leveraging Pre-trained Checkpoints for Sequence Generation Taskscited by 182
  17. 2023Unsafe Diffusion: On the Generation of Unsafe Images and Hateful Memes From Text-To-Image Modelscited by 171
  18. 2023Holistic Evaluation of Text-To-Image Modelscited by 160
  19. 2019Deeper and Wider Siamese Networks for Real-Time Visual Trackingcited by 156
  20. 2020Augmented SBERT: Data Augmentation Method for Improving Bi-Encoders for Pairwise Sentence Scoring Taskscited by 136
  21. 2021TSDAE: Using Transformer-based Sequential Denoising Auto-Encoder for Unsupervised Sentence Embedding Learningcited by 116
  22. 2017Learned in Translation: Contextualized Word Vectorscited by 104
  23. 2018A Twofold Siamese Network for Real-Time Object Trackingcited by 62
  24. 2023SurrogatePrompt: Bypassing the Safety Filter of Text-to-Image Models via Substitutioncited by 32
  25. 2021Mixed Cross Entropy Loss for Neural Machine Translationcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 15; 2025: 27; 2026: 19.

61 papers.

Sort:

By area:

  1. 2024SAFREE: Training-Free and Adaptive Guard for Safe Text-to-Image And Video Generationcited by 125
  2. 2024Navigating the Risks: A Survey of Security, Privacy, and Ethics Threats in LLM-Based Agentscited by 63
  3. 2024VBench++: Comprehensive and Versatile Benchmark Suite for Video Generative Modelscited by 42
  4. 2024Meta-Unlearning on Diffusion Models: Preventing Relearning Unlearned Conceptscited by 39
  5. 2024Adversarial Attacks and Defenses on Text-to-Image Diffusion Models: A Surveycited by 34
  6. 2024Attacks and Defenses for Generative Diffusion Models: A Comprehensive Surveycited by 32
  7. 2024Jailbreak Attacks and Defenses against Multimodal Generative Models: A Surveycited by 31
  8. 2024Espresso: Robust Concept Filtering in Text-to-Image Modelscited by 14
  9. 2025Concept Corrector: Erase concepts on the fly for text-to-image diffusion modelscited by 11
  10. 2024AdvI2I: Adversarial Image Attack on Image-to-Image Diffusion modelscited by 10
  11. 2025Erasing Concepts, Steering Generations: A Comprehensive Survey of Concept Suppressioncited by 10
  12. 2025A Comprehensive Survey on Concept Erasure in Text-to-Image Diffusion Modelscited by 9
38 more
  1. 2025Token-Level Constraint Boundary Search for Jailbreaking Text-to-Image Modelscited by 9
  2. 2025Red-Teaming Text-to-Image Systems by Rule-based Preference Modelingcited by 8
  3. 2025Towards Resilient Safety-driven Unlearning for Diffusion Models against Downstream Fine-tuningcited by 8
  4. 2025PLA: Prompt Learning Attack against Text-to-Image Generative Modelscited by 8
  5. 2024AEIOU: A Unified Defense Framework against NSFW Prompts in Text-to-Image Modelscited by 7
  6. 2025Mitigating Sexual Content Generation via Embedding Distortion in Text-conditioned Diffusion Modelscited by 7
  7. 2025TarPro: Targeted Protection against Malicious Image Editingcited by 7
  8. 2025SafeGuider: Robust and Practical Content Safety Control for Text-to-Image Modelscited by 6
  9. 2025Training-Free Safe Text Embedding Guidance for Text-to-Image Diffusion Modelscited by 6
  10. 2025Plug-and-Play Interpretable Responsible Text-to-Image Generation via Dual-Space Multi-facet Concept Controlcited by 5
  11. 2025Dynamic Optimization and Safety Indicator Injection for Jailbreaking Text-to-Image Models with Multimodal Safety Filterscited by 5
  12. 2025Universally Unfiltered and Unseen:Input-Agnostic Multimodal Jailbreaks against Text-to-Image Model Safeguardscited by 5
  13. 2024Safe Text-to-Image Generation: Simply Sanitize the Prompt Embeddingcited by 4
  14. 2025PurifyGen: A Risk-Discrimination and Semantic-Purification Model for Safe Text-to-Image Generationcited by 4
  15. 2024Buster: Implanting Semantic Backdoor into Text Encoder to Mitigate NSFW Content Generationcited by 3
  16. 2024From Noise to Nuance: Advances in Deep Generative Image Modelscited by 3
  17. 2025Wukong Framework for Not Safe For Work Detection in Text-to-Image systemscited by 3
  18. 2025Seeing It Before It Happens: In-Generation NSFW Detection for Diffusion-Based Text-to-Image Modelscited by 3
  19. 2026On the Adversarial Robustness of Multimodal LLM Judgescited by 3
  20. 2024Antelope: Potent and Concealed Jailbreak Attack Strategycited by 2
  21. 2025Multimodal Prompt Decoupling Attack on the Safety Filters in Text-to-Image Modelscited by 2
  22. 2025SafeEditor: Unified MLLM for Efficient Post-hoc T2I Safety Editingcited by 2
  23. 2025Value-Aligned Prompt Moderation via Zero-Shot Agentic Rewriting for Safe Image Generationcited by 2
  24. 2026Selective Fine-Tuning for Targeted and Robust Concept Unlearningcited by 2
  25. 2026JANUS: A Lightweight Framework for Jailbreaking Text-to-Image Models via Distribution Optimizationcited by 2
  26. 2025Do Concept Replacement Techniques Really Erase Unacceptable Concepts?cited by 1
  27. 2025Patronus: Safeguarding Text-to-Image Models against White-Box Adversariescited by 1
  28. 2026Harnessing Hyperbolic Geometry for Harmful Prompt Detection and Sanitizationcited by 1
  29. 2026TrajShield: Trajectory-Level Safety Mediation for Defending Text-to-Video Models Against Jailbreak Attackscited by 1
  30. 2024Finding a Wolf in Sheep's Clothing: Combating Adversarial Text-To-Image Prompts with Text Summarizationcited by 0
  31. 2025SafeCtrl: Region-Based Safety Control for Text-to-Image Diffusion via Detect-Then-Suppresscited by 0
  32. 2025Security Risk of Misalignment between Text and Image in Multi-modal Modelcited by 0
  33. 2025SP-Guard: Selective Prompt-adaptive Guidance for Safe Text-to-Image Generationcited by 0
  34. 2025CAHS-Attack: CLIP-Aware Heuristic Search Attack Method for Stable Diffusioncited by 0
  35. 2025Metaphor-based Jailbreak Attacks on Text-to-Image Modelscited by 0
  36. 2026SPOT: Selective Prompt Projection via Total Variation for Inference-Only Safe Text-to-Image Generationcited by 0
  37. 2026IdentityGuard: Context-Aware Restriction and Provenance for Personalized Synthesiscited by 0
  38. 2026EDGE-Shield: Efficient Denoising-staGE Shield for Violative Content Filtering via Scalable Reference-Based Matchingcited by 0

50 most cited of 61 All 61 (JSON)