# When There Is No Decoder: Removing Watermarks from Stable Diffusion Models in a No-box Setting

arXiv: https://arxiv.org/abs/2507.03646 · HTML: https://mlsanity.com/paper/2507.03646/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models.
- Builds on 24 of its 30 references that are in the map.
- First posted on arXiv 4 Jul 2025.

## Placed in

- Diffusion models / [Robustness, privacy and adversarial aspects](https://mlsanity.com/diffusion/robustness-privacy-and-adversarial-aspects/)
- LLM safety and security / [Watermarking generated content and models](https://mlsanity.com/safety/watermarking-generated-content-and-models/)

## Builds on (most cited first, 24 of 24)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Understanding Black-box Predictions via Influence Functions](https://mlsanity.com/paper/1703.04730/) (2017)
- [HiDDeN: Hiding Data With Deep Networks](https://mlsanity.com/paper/1807.09937/) (2018)
- [HopSkipJumpAttack: A Query-Efficient Decision-Based Attack](https://mlsanity.com/paper/1904.02144/) (2019)
- [The Stable Signature: Rooting Watermarks in Latent Diffusion Models](https://mlsanity.com/paper/2303.15435/) (2023)
- [Provable Robust Watermarking for AI-Generated Text](https://mlsanity.com/paper/2306.17439/) (2023)
- [StegaStamp: Invisible Hyperlinks in Physical Photographs](https://mlsanity.com/paper/1904.05343/) (2019)
- [Efficient Frequency Domain-based Transformers for High-Quality Image Deblurring](https://mlsanity.com/paper/2211.12250/) (2022)
- [Invisible Image Watermarks Are Provably Removable Using Generative AI](https://mlsanity.com/paper/2306.01953/) (2023)
- [WAVES: Benchmarking the Robustness of Image Watermarks](https://mlsanity.com/paper/2401.08573/) (2024)
- [Evading Watermark based Detection of AI-Generated Content](https://mlsanity.com/paper/2305.03807/) (2023)
- [Unveiling Security, Privacy, and Ethical Concerns of ChatGPT](https://mlsanity.com/paper/2307.14192/) (2023)
- [Leveraging Optimization for Adaptive Attacks on Image Watermarks](https://mlsanity.com/paper/2309.16952/) (2023)
- [Watermarking Diffusion Model](https://mlsanity.com/paper/2305.12502/) (2023)
- [A Transfer Attack to Image Watermarks](https://mlsanity.com/paper/2403.15365/) (2024)
- [Generative Watermarking Against Unauthorized Subject-Driven Image Synthesis](https://mlsanity.com/paper/2306.07754/) (2023)
- [Intellectual Property Protection of Diffusion Models via the Watermark Diffusion Process](https://mlsanity.com/paper/2306.03436/) (2023)
- [Securing Deep Generative Models with Universal Adversarial Signature](https://mlsanity.com/paper/2305.16310/) (2023)
- [Certifiably Robust Image Watermark](https://mlsanity.com/paper/2407.04086/) (2024)
- [Assessing the Efficacy of Invisible Watermarks in AI-Generated Medical Images](https://mlsanity.com/paper/2402.03473/) (2024)

## Built on by (most cited first, 3 of 3)

- [Secure and Robust Watermarking for AI-generated Images: A Comprehensive Survey](https://mlsanity.com/paper/2510.02384/) (2025)
- [MarkSweep: A No-box Removal Attack on AI-Generated Image Watermarking via Noise Intensification and Frequency-aware Denoising](https://mlsanity.com/paper/2602.15364/) (2026)
- [SPFM-Net: Semantic-Prior-Guided Frequency-Constrained Mamba for Invisible Watermark Attack](https://mlsanity.com/paper/2607.27811/) (2026)

## Abstract (arXiv)

Watermarking has emerged as a promising solution to counter harmful or deceptive AI-generated content by embedding hidden identifiers that trace content origins. However, the robustness of current watermarking techniques is still largely unexplored, raising critical questions about their effectiveness against adversarial attacks. To address this gap, we examine the robustness of model-specific watermarking, where watermark embedding is integrated with text-to-image generation in models like latent diffusion models. We introduce three attack strategies: edge prediction-based, box blurring, and fine-tuning-based attacks in a no-box setting, where an attacker does not require access to the ground-truth watermark decoder. Our findings reveal that while model-specific watermarking is resilient against basic evasion attempts, such as edge prediction, it is notably vulnerable to blurring and fine-tuning-based attacks. Our best-performing attack achieves a reduction in watermark detection accuracy to approximately 47.92%. Additionally, we perform an ablation study on factors like message length, kernel size and decoder depth, identifying critical parameters influencing the fine-tuning attack's success. Finally, we assess several advanced watermarking defenses, finding that even the most robust methods, such as multi-label smoothing, result in watermark extraction accuracy that falls below an acceptable level when subjected to our no-box attacks.