mlsanityData to 7 Oct 2026

Latent Guard: a Safety Framework for Text-to-image Generation

arXiv 2404.08031 ↗First posted 11 Apr 2024cs.CV cs.AI cs.LGcited by 68

Not on the map. Why?

Cited by 29 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

With the ability to generate high-quality images, text-to-image (T2I) models can be exploited for creating inappropriate content. To prevent misuse, existing safety measures are either based on text blacklists, which can be easily circumvented, or harmful content classification, requiring large datasets for training and offering low flexibility. Hence, we propose Latent Guard, a framework designed to improve safety measures in text-to-image generation. Inspired by blacklist-based approaches, Latent Guard learns a latent space on top of the T2I model's text encoder, where it is possible to check the presence of harmful concepts in the input text embeddings. Our proposed framework is composed of a data generation pipeline specific to the task using large language models, ad-hoc architectural components, and a contrastive learning strategy to benefit from the generated data. The effectiveness of our method is verified on three datasets and against four baselines. Code and data will be shared at https://latentguard.github.io/.

Key facts. Cited by 68 papers in the map (data through 7 Oct 2026). Builds on 41 of its 54 references that are in the map. First posted on arXiv 11 Apr 2024.

Builds on

41 of 54 references.

References · 41

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  5. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  6. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  8. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,322
  9. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  10. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  11. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  12. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
29 more
  1. 2023Universal and Transferable Adversarial Attacks on Aligned Language Modelscited by 2,914
  2. 2020Supervised Contrastive Learningcited by 2,803
  3. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  4. 2023Llama Guard: LLM-based Input-Output Safeguard for Human-AI Conversationscited by 1,124
  5. 2021Vector Quantized Diffusion Model for Text-to-Image Synthesiscited by 773
  6. 2023Erasing Concepts from Diffusion Modelscited by 530
  7. 2022Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Modelscited by 501
  8. 2022A Holistic Approach to Undesired Content Detection in the Real Worldcited by 337
  9. 2023Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discoverycited by 329
  10. 2019DM-GAN: Dynamic Memory Generative Adversarial Networks for Text-to-Image Synthesiscited by 308
  11. 2023MM-SafetyBench: A Benchmark for Safety Evaluation of Multimodal Large Language Modelscited by 255
  12. 2023Ring-A-Bell! How Reliable are Concept Removal Methods for Diffusion Models?cited by 239
  13. 2023MMA-Diffusion: MultiModal Attack on Diffusion Modelscited by 184
  14. 2023SneakyPrompt: Jailbreaking Text-to-image Generative Modelscited by 175
  15. 2023Unsafe Diffusion: On the Generation of Unsafe Images and Hateful Memes From Text-To-Image Modelscited by 171
  16. 2023Prompting4Debugging: Red-Teaming Text-to-Image Diffusion Models by Finding Problematic Promptscited by 161
  17. 2022Can Machines Help Us Answering Question 16 in Datasheets, and In Turn Reflecting on Inappropriate Content?cited by 150
  18. 2021Between words and characters: A Brief History of Open-Vocabulary Modeling and Tokenization in NLPcited by 130
  19. 2023A Pilot Study of Query-Free Adversarial Attack against Stable Diffusioncited by 75
  20. 2023Self-Discovering Interpretable Diffusion Latent Directions for Responsible Text-to-Image Generationcited by 69
  21. 2024SynthCLIP: Are We Ready for a Fully Synthetic CLIP Training?cited by 64
  22. 2022Discovering the Hidden Vocabulary of DALLE-2cited by 42
  23. 2022Adversarial Attacks on Image Generation With Made-Up Wordscited by 29
  24. 2023Discovering Failure Modes of Text-guided Diffusion Models via Adversarial Searchcited by 25
  25. 2023IMMA: Immunizing text-to-image Models against Malicious Adaptationcited by 17
  26. 2023Safe-CLIP: Removing NSFW Concepts from Vision-and-Language Modelscited by 11
  27. 2024A Survey on Responsible Generative AI: What to Generate and What Notcited by 8
  28. 2023Distilling Adversarial Prompts from Safety Benchmarks: Report for the Adversarial Nibbler Challengecited by 6
  29. 2024Discovering Universal Semantic Triggers for Text-to-Image Synthesiscited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 17; 2025: 30; 2026: 21.

68 papers.

Sort:

By area:

  1. 2024SAFREE: Training-Free and Adaptive Guard for Safe Text-to-Image And Video Generationcited by 125
  2. 2024LLMs Meet Multimodal Generation and Editing: A Surveycited by 35
  3. 2024Adversarial Attacks and Defenses on Text-to-Image Diffusion Models: A Surveycited by 34
  4. 2024Jailbreak Attacks and Defenses against Multimodal Generative Models: A Surveycited by 31
  5. 2025ShieldGemma 2: Robust and Tractable Image Content Moderationcited by 27
  6. 2024How to Trace Latent Generative Model Generated Images without Artificial Watermark?cited by 25
  7. 2025CURE: Concept Unlearning via Orthogonal Representation Editing in Diffusion Modelscited by 24
  8. 2025Training-Free Safe Denoisers for Safe Use of Diffusion Modelscited by 19
  9. 2024AlignGuard: Scalable Safety Alignment for Text-to-Image Generationcited by 17
  10. 2025PromptGuard: Soft Prompt-Guided Unsafe Content Moderation for Text-to-Image Modelscited by 16
  11. 2025Concept Corrector: Erase concepts on the fly for text-to-image diffusion modelscited by 11
  12. 2024SteerDiff: Steering towards Safe Text-to-Image Diffusion Modelscited by 10
38 more
  1. 2024AdvI2I: Adversarial Image Attack on Image-to-Image Diffusion modelscited by 10
  2. 2025Erasing Concepts, Steering Generations: A Comprehensive Survey of Concept Suppressioncited by 10
  3. 2025A Comprehensive Survey on Concept Erasure in Text-to-Image Diffusion Modelscited by 9
  4. 2025Towards Resilient Safety-driven Unlearning for Diffusion Models against Downstream Fine-tuningcited by 8
  5. 2025PLA: Prompt Learning Attack against Text-to-Image Generative Modelscited by 8
  6. 2026Loop as a Bridge: Can Looped Transformers Truly Link Representation Space and Natural Language Outputs?cited by 8
  7. 2024Not Just Text: Uncovering Vision Modality Typographic Threats in Image Generation Modelscited by 7
  8. 2024AEIOU: A Unified Defense Framework against NSFW Prompts in Text-to-Image Modelscited by 7
  9. 2025Mitigating Sexual Content Generation via Embedding Distortion in Text-conditioned Diffusion Modelscited by 7
  10. 2025TarPro: Targeted Protection against Malicious Image Editingcited by 7
  11. 2025CRCE: Coreference-Retention Concept Erasure in Text-to-Image Diffusion Modelscited by 6
  12. 2025Training-Free Safe Text Embedding Guidance for Text-to-Image Diffusion Modelscited by 6
  13. 2025DeContext as Defense: Safe Image Editing in Diffusion Transformerscited by 6
  14. 2024Class-RAG: Real-Time Content Moderation with Retrieval Augmented Generationcited by 5
  15. 2024Towards Effective Discrimination Testing for Generative AIcited by 5
  16. 2025OVERT: A Benchmark for Over-Refusal Evaluation on Text-to-Image Modelscited by 5
  17. 2024Safe Text-to-Image Generation: Simply Sanitize the Prompt Embeddingcited by 4
  18. 2024Towards Action Hijacking of Large Language Model-based Agentcited by 4
  19. 2025EAR: Erasing Concepts from Unified Autoregressive Modelscited by 4
  20. 2026Neighbor-Aware Localized Concept Erasure in Text-to-Image Diffusion Modelscited by 4
  21. 2025Responsible Diffusion: A Comprehensive Survey on Safety, Ethics, and Trust in Diffusion Modelscited by 3
  22. 2025CGCE: Classifier-Guided Concept Erasure in Generative Modelscited by 3
  23. 2026Conditioned Activation Transport for T2I Safety Steeringcited by 3
  24. 2026On the Adversarial Robustness of Multimodal LLM Judgescited by 3
  25. 2025Beyond External Monitors: Enhancing Transparency of Large Language Models for Easier Monitoringcited by 2
  26. 2025A Systematic Review of Open Datasets Used in Text-to-Image (T2I) Gen AI Model Safetycited by 2
  27. 2025An h-space Based Adversarial Attack for Protection Against Few-shot Personalizationcited by 2
  28. 2025Multimodal Prompt Decoupling Attack on the Safety Filters in Text-to-Image Modelscited by 2
  29. 2025SafeEditor: Unified MLLM for Efficient Post-hoc T2I Safety Editingcited by 2
  30. 2025SPQR: A Multi-Dimensional Benchmark for Safety Alignment under Benign Model Adaptationcited by 2
  31. 2026ActErase: A Training-Free Paradigm for Precise Concept Erasure via Activation Redirectioncited by 2
  32. 2026Do Models Share Safety Representations? Cross-Model Steering for Safe Visual Generationcited by 2
  33. 2024Safety Without Semantic Disruptions: Editing-free Safe Image Generation via Context-preserving Dual Latent Reconstructioncited by 1
  34. 2024Red-Teaming Text-to-Image Models via In-Context Experience Replay and Semantic-Preserving Prompt Rewritingcited by 1
  35. 2025Safe Vision-Language Models via Unsafe Weights Manipulationcited by 1
  36. 2025Personalized Safety Alignment for Text-to-Image Diffusion Modelscited by 1
  37. 2026Harnessing Hyperbolic Geometry for Harmful Prompt Detection and Sanitizationcited by 1
  38. 2026FlowErase-RL: Rethinking Concept Erasure as Reward Optimization in Flow Matching Modelscited by 1

50 most cited of 68 All 68 (JSON)