# DiffGuard: Text-Based Safety Checker for Diffusion Models

arXiv: https://arxiv.org/abs/2412.00064 · HTML: https://mlsanity.com/paper/2412.00064/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models.
- Builds on 14 of its 15 references that are in the map.
- First posted on arXiv 25 Nov 2024.

## Placed in

- Diffusion models / [Robustness, privacy and adversarial aspects](https://mlsanity.com/diffusion/robustness-privacy-and-adversarial-aspects/)

## Builds on (most cited first, 14 of 14)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [All are Worth Words: A ViT Backbone for Diffusion Models](https://mlsanity.com/paper/2209.12152/) (2022)
- [Sora: A Review on Background, Technology, Limitations, and Opportunities of Large Vision Models](https://mlsanity.com/paper/2402.17177/) (2024)
- [Red-Teaming the Stable Diffusion Safety Filter](https://mlsanity.com/paper/2210.04610/) (2022)
- [MMA-Diffusion: MultiModal Attack on Diffusion Models](https://mlsanity.com/paper/2311.17516/) (2023)
- [SneakyPrompt: Jailbreaking Text-to-image Generative Models](https://mlsanity.com/paper/2305.12082/) (2023)
- [Unsafe Diffusion: On the Generation of Unsafe Images and Hateful Memes From Text-To-Image Models](https://mlsanity.com/paper/2305.13873/) (2023)
- [Can Machines Help Us Answering Question 16 in Datasheets, and In Turn Reflecting on Inappropriate Content?](https://mlsanity.com/paper/2202.06675/) (2022)
- [Large Language Models Are Zero-Shot Text Classifiers](https://mlsanity.com/paper/2312.01044/) (2023)
- [Building Efficient Universal Classifiers with Natural Language Inference](https://mlsanity.com/paper/2312.17543/) (2023)

## Built on by (most cited first, 10 of 10)

- [JANUS: A Lightweight Framework for Jailbreaking Text-to-Image Models via Distribution Optimization](https://mlsanity.com/paper/2603.21208/) (2026)
- [When Understanding Becomes a Risk: Authenticity and Safety Risks in the Emerging Image Generation Paradigm](https://mlsanity.com/paper/2603.24079/) (2026)
- [Harnessing Hyperbolic Geometry for Harmful Prompt Detection and Sanitization](https://mlsanity.com/paper/2604.06285/) (2026)
- [Evaluation without Generation: Non-Generative Assessment of Harmful Model Specialization with Applications to CSAM](https://mlsanity.com/paper/2604.25119/) (2026)
- [Between Safe Boundaries: Exploiting Temporal Consistency for Jailbreaking Text-To-Video Generation Models](https://mlsanity.com/paper/2607.17279/) (2026)
- [Metaphor-based Jailbreak Attacks on Text-to-Image Models](https://mlsanity.com/paper/2512.10766/) (2025)
- [Modular Energy Steering for Safe Text-to-Image Generation with Foundation Models](https://mlsanity.com/paper/2604.02265/) (2026)
- [AEGIS: A Mechanism-Guided Defense against Visual Synonym Jailbreaks in Text-to-Image Models](https://mlsanity.com/paper/2607.06120/) (2026)
- [Architectural Backdoors in Vision-Language Model Supply Chains via Representation Steering](https://mlsanity.com/paper/2607.25479/) (2026)
- [Mind the Gap: Zero-Query Jailbreaks via Filter-Generator Discrepancy in Text-to-Image Systems](https://mlsanity.com/paper/2608.00973/) (2026)

## Abstract (arXiv)

Recent advances in Diffusion Models have enabled the generation of images from text, with powerful closed-source models like DALL-E and Midjourney leading the way. However, open-source alternatives, such as StabilityAI's Stable Diffusion, offer comparable capabilities. These open-source models, hosted on Hugging Face, come equipped with ethical filter protections designed to prevent the generation of explicit images. This paper reveals first their limitations and then presents a novel text-based safety filter that outperforms existing solutions. Our research is driven by the critical need to address the misuse of AI-generated content, especially in the context of information warfare. DiffGuard enhances filtering efficacy, achieving a performance that surpasses the best existing filters by over 14%.