# DiffProtect: Generate Adversarial Examples with Diffusion Models for Facial Privacy Protection

arXiv: https://arxiv.org/abs/2305.13625 · HTML: https://mlsanity.com/paper/2305.13625/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models.
- Builds on 48 of its 77 references that are in the map.
- First posted on arXiv 23 May 2023.

## Placed in

- Diffusion models / [Robustness, privacy and adversarial aspects](https://mlsanity.com/diffusion/robustness-privacy-and-adversarial-aspects/)

## Builds on (most cited first, 30 of 48)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Improved Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2102.09672/) (2021)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation](https://mlsanity.com/paper/1808.00897/) (2018)
- [MaskGAN: Towards Diverse and Interactive Facial Image Manipulation](https://mlsanity.com/paper/1907.11922/) (2019)
- [Feature Squeezing: Detecting Adversarial Examples in Deep Neural Networks](https://mlsanity.com/paper/1704.01155/) (2017)
- [Diffusion Models for Adversarial Purification](https://mlsanity.com/paper/2205.07460/) (2022)
- [Diffusion Autoencoders: Toward a Meaningful and Decodable Representation](https://mlsanity.com/paper/2111.15640/) (2021)
- [Generating Adversarial Examples with Adversarial Networks](https://mlsanity.com/paper/1801.02610/) (2018)
- [Improving Robustness using Generated Data](https://mlsanity.com/paper/2110.09468/) (2021)
- [Better Diffusion Models Further Improve Adversarial Training](https://mlsanity.com/paper/2302.04638/) (2023)
- [Fixing Data Augmentation to Improve Adversarial Robustness](https://mlsanity.com/paper/2103.01946/) (2021)
- [A study of the effect of JPG compression on adversarial images](https://mlsanity.com/paper/1608.00853/) (2016)
- [High-Fidelity GAN Inversion for Image Attribute Editing](https://mlsanity.com/paper/2109.06590/) (2021)

## Built on by (most cited first, 30 of 35)

- [Toward effective protection against diffusion based mimicry through score distillation](https://mlsanity.com/paper/2311.12832/) (2023)
- [On the Design Fundamentals of Diffusion Models: A Survey](https://mlsanity.com/paper/2306.04542/) (2023)
- [Transferable Adversarial Facial Images for Privacy Protection](https://mlsanity.com/paper/2408.01428/) (2024)
- [Prompt-Agnostic Adversarial Perturbation for Customized Diffusion Models](https://mlsanity.com/paper/2408.10571/) (2024)
- [Instant Adversarial Purification with Adversarial Consistency Distillation](https://mlsanity.com/paper/2408.17064/) (2024)
- [Adv-CPG: A Customized Portrait Generation Framework with Facial Adversarial Attacks](https://mlsanity.com/paper/2503.08269/) (2025)
- [A Gray-box Attack against Latent Diffusion Model-based Image Editing by Posterior Collapse](https://mlsanity.com/paper/2408.10901/) (2024)
- [A Survey of Emerging Applications of Diffusion Probabilistic Models in MRI](https://mlsanity.com/paper/2311.11383/) (2023)
- [Enhancing Facial Privacy Protection via Weakening Diffusion Purification](https://mlsanity.com/paper/2503.10350/) (2025)
- [DCT-Shield: A Robust Frequency Domain Defense against Malicious Image Editing](https://mlsanity.com/paper/2504.17894/) (2025)
- [T2VAttack: Adversarial Attack on Text-to-Video Diffusion Models](https://mlsanity.com/paper/2512.23953/) (2025)
- [Natural Language Induced Adversarial Images](https://mlsanity.com/paper/2410.08620/) (2024)
- [Concept-as-Tree: A Controllable Synthetic Data Framework Makes Stronger Personalized VLMs](https://mlsanity.com/paper/2503.12999/) (2025)
- [DIP-Watermark: A Double Identity Protection Method Based on Robust Adversarial Watermark](https://mlsanity.com/paper/2404.14693/) (2024)
- [SCA: Improve Semantic Consistent in Unrestricted Adversarial Attacks via DDPM Inversion](https://mlsanity.com/paper/2410.02240/) (2024)
- [Adversarial Attacks for Good: A Survey of Proactive Protection across the Visual Content Lifecycle](https://mlsanity.com/paper/2608.04314/) (2026)
- [MMAD-Purify: A Precision-Optimized Framework for Efficient and Scalable Multi-Modal Attacks](https://mlsanity.com/paper/2410.14089/) (2024)
- [ErasableMask: A Robust and Erasable Privacy Protection Scheme against Black-box Face Recognition Models](https://mlsanity.com/paper/2412.17038/) (2024)
- [Machine Pareidolia: Protecting Facial Image with Emotional Editing](https://mlsanity.com/paper/2603.03665/) (2026)
- [Adv-TGD: Adversarial Text-Guided Diffusion for Face Recognition Impersonation Attacks](https://mlsanity.com/paper/2606.11615/) (2026)
- [From text to multimodal: a survey of adversarial example generation in question answering systems](https://mlsanity.com/paper/2312.16156/) (2023)
- [DataCook: Crafting Anti-Adversarial Examples for Healthcare Data Copyright Protection](https://mlsanity.com/paper/2403.17755/) (2024)
- [Controllable Adversarial Makeup for Privacy via Text-Guided Diffusion](https://mlsanity.com/paper/2503.10549/) (2025)
- [Computation-Efficient and Recognition-Friendly 3D Point Cloud Privacy Protection](https://mlsanity.com/paper/2503.15818/) (2025)
- [DP-TRAE: A Dual-Phase Merging Transferable Reversible Adversarial Example for Image Privacy Protection](https://mlsanity.com/paper/2505.06860/) (2025)
- [Unmasking Synthetic Realities in Generative AI: A Comprehensive Review of Adversarially Robust Deepfake Detection Systems](https://mlsanity.com/paper/2507.21157/) (2025)
- [Diffusion Guided Adversarial State Perturbations in Reinforcement Learning](https://mlsanity.com/paper/2511.07701/) (2025)
- [SIDeR: Semantic Identity Decoupling for Unrestricted Face Privacy](https://mlsanity.com/paper/2602.04994/) (2026)
- [DiffInf: Influence-Guided Diffusion for Supervision Alignment in Facial Attribute Learning](https://mlsanity.com/paper/2603.06399/) (2026)
- [Personalized Face Privacy Protection From a Single Image](https://mlsanity.com/paper/2605.19032/) (2026)

## Abstract (arXiv)

The increasingly pervasive facial recognition (FR) systems raise serious concerns about personal privacy, especially for billions of users who have publicly shared their photos on social media. Several attempts have been made to protect individuals from being identified by unauthorized FR systems utilizing adversarial attacks to generate encrypted face images. However, existing methods suffer from poor visual quality or low attack success rates, which limit their utility. Recently, diffusion models have achieved tremendous success in image generation. In this work, we ask: can diffusion models be used to generate adversarial examples to improve both visual quality and attack performance? We propose DiffProtect, which utilizes a diffusion autoencoder to generate semantically meaningful perturbations on FR systems. Extensive experiments demonstrate that DiffProtect produces more natural-looking encrypted images than state-of-the-art methods while achieving significantly higher attack success rates, e.g., 24.5% and 25.1% absolute improvements on the CelebA-HQ and FFHQ datasets.