# Safe and Reliable Diffusion Models via Subspace Projection

arXiv: https://arxiv.org/abs/2503.16835 · HTML: https://mlsanity.com/paper/2503.16835/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models.
- Builds on 25 of its 33 references that are in the map.
- First posted on arXiv 21 Mar 2025.

## Placed in

- Diffusion models / [Robustness, privacy and adversarial aspects](https://mlsanity.com/diffusion/robustness-privacy-and-adversarial-aspects/)

## Builds on (most cited first, 25 of 25)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [Reproducible scaling laws for contrastive language-image learning](https://mlsanity.com/paper/2212.07143/) (2022)
- [Improved Techniques for Training Score-Based Generative Models](https://mlsanity.com/paper/2006.09011/) (2020)
- [Erasing Concepts from Diffusion Models](https://mlsanity.com/paper/2303.07345/) (2023)
- [Safe Latent Diffusion: Mitigating Inappropriate Degeneration in Diffusion Models](https://mlsanity.com/paper/2211.05105/) (2022)
- [Unified Concept Editing in Diffusion Models](https://mlsanity.com/paper/2308.14761/) (2023)
- [Ablating Concepts in Text-to-Image Diffusion Models](https://mlsanity.com/paper/2303.13516/) (2023)
- [MACE: Mass Concept Erasure in Diffusion Models](https://mlsanity.com/paper/2403.06135/) (2024)
- [Forget-Me-Not: Learning to Forget in Text-to-Image Diffusion Models](https://mlsanity.com/paper/2303.17591/) (2023)
- [Red-Teaming the Stable Diffusion Safety Filter](https://mlsanity.com/paper/2210.04610/) (2022)
- [Unsafe Diffusion: On the Generation of Unsafe Images and Hateful Memes From Text-To-Image Models](https://mlsanity.com/paper/2305.13873/) (2023)
- [One-Dimensional Adapter to Rule Them All: Concepts, Diffusion Models and Erasing Applications](https://mlsanity.com/paper/2312.16145/) (2023)
- [Measuring Style Similarity in Diffusion Models](https://mlsanity.com/paper/2404.01292/) (2024)
- [The ArtBench Dataset: Benchmarking Generative Models with Artworks](https://mlsanity.com/paper/2206.11404/) (2022)
- [Learn to Forget: Machine Unlearning via Neuron Masking](https://mlsanity.com/paper/2003.10933/) (2020)
- [Localizing and Editing Knowledge in Text-to-Image Generative Models](https://mlsanity.com/paper/2310.13730/) (2023)
- [Precise, Fast, and Low-cost Concept Erasure in Value Space: Orthogonal Complement Matters](https://mlsanity.com/paper/2412.06143/) (2024)
- [On Mechanistic Knowledge Localization in Text-to-Image Generative Models](https://mlsanity.com/paper/2405.01008/) (2024)
- [Espresso: Robust Concept Filtering in Text-to-Image Models](https://mlsanity.com/paper/2404.19227/) (2024)
- [Don't Forget Too Much: Towards Machine Unlearning on Feature Level](https://mlsanity.com/paper/2406.10951/) (2024)

## Built on by (most cited first, 7 of 7)

- [Set You Straight: Auto-Steering Denoising Trajectories to Sidestep Unwanted Concepts](https://mlsanity.com/paper/2504.12782/) (2025)
- [Erasing Concepts, Steering Generations: A Comprehensive Survey of Concept Suppression](https://mlsanity.com/paper/2505.19398/) (2025)
- [Erased or Dormant? Rethinking Concept Erasure Through Reversibility](https://mlsanity.com/paper/2505.16174/) (2025)
- [SafeMo: Linguistically Grounded Unlearning for Trustworthy Text-to-Motion Generation](https://mlsanity.com/paper/2601.00590/) (2026)
- [Concept Unlearning by Modeling Key Steps of Diffusion Process](https://mlsanity.com/paper/2507.06526/) (2025)
- [What Lurks Within? Concept Auditing for Shared Diffusion Models at Scale](https://mlsanity.com/paper/2504.14815/) (2025)
- [ACE: Attentional Concept Erasure in Diffusion Models](https://mlsanity.com/paper/2504.11850/) (2025)

## Abstract (arXiv)

Large-scale text-to-image (T2I) diffusion models have revolutionized image generation, enabling the synthesis of highly detailed visuals from textual descriptions. However, these models may inadvertently generate inappropriate content, such as copyrighted works or offensive images. While existing methods attempt to eliminate specific unwanted concepts, they often fail to ensure complete removal, allowing the concept to reappear in subtle forms. For instance, a model may successfully avoid generating images in Van Gogh's style when explicitly prompted with 'Van Gogh', yet still reproduce his signature artwork when given the prompt 'Starry Night'. In this paper, we propose SAFER, a novel and efficient approach for thoroughly removing target concepts from diffusion models. At a high level, SAFER is inspired by the observed low-dimensional structure of the text embedding space. The method first identifies a concept-specific subspace Sc associated with the target concept c. It then projects the prompt embeddings onto the complementary subspace of Sc, effectively erasing the concept from the generated images. Since concepts can be abstract and difficult to fully capture using natural language alone, we employ textual inversion to learn an optimized embedding of the target concept from a reference image. This enables more precise subspace estimation and enhances removal performance. Furthermore, we introduce a subspace expansion strategy to ensure comprehensive and robust concept erasure. Extensive experiments demonstrate that SAFER consistently and effectively erases unwanted concepts from diffusion models while preserving generation quality.