# Diff-Privacy: Diffusion-based Face Privacy Protection

arXiv: https://arxiv.org/abs/2309.05330 · HTML: https://mlsanity.com/paper/2309.05330/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 65 references that are in the map.
- First posted on arXiv 11 Sep 2023.

## Builds on (most cited first, 30 of 40)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Generative Modeling by Estimating Gradients of the Data Distribution](https://mlsanity.com/paper/1907.05600/) (2019)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022)
- [eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers](https://mlsanity.com/paper/2211.01324/) (2022)
- [Blended Diffusion for Text-driven Editing of Natural Images](https://mlsanity.com/paper/2111.14818/) (2021)
- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [FreeDoM: Training-Free Energy-Guided Conditional Diffusion Model](https://mlsanity.com/paper/2303.09833/) (2023)
- [Inversion-Based Style Transfer with Diffusion Models](https://mlsanity.com/paper/2211.13203/) (2022)
- [More Control for Free! Image Synthesis with Semantic Diffusion Guidance](https://mlsanity.com/paper/2112.05744/) (2021)
- [Generative Diffusion Prior for Unified Image Restoration and Enhancement](https://mlsanity.com/paper/2304.01247/) (2023)
- [EGSDE: Unpaired Image-to-Image Translation via Energy-Guided Stochastic Differential Equations](https://mlsanity.com/paper/2207.06635/) (2022)
- [Diffusion-based Image Translation using Disentangled Style and Content Representation](https://mlsanity.com/paper/2209.15264/) (2022)
- [DeepPrivacy: A Generative Adversarial Network for Face Anonymization](https://mlsanity.com/paper/1909.04538/) (2019)
- [CIAGAN: Conditional Identity Anonymization Generative Adversarial Networks](https://mlsanity.com/paper/2005.09544/) (2020)
- [ProSpect: Prompt Spectrum for Attribute-Aware Personalization of Diffusion Models](https://mlsanity.com/paper/2305.16225/) (2023)
- [Natural and Effective Obfuscation by Head Inpainting](https://mlsanity.com/paper/1711.09001/) (2017)
- [DeepPrivacy2: Towards Realistic Full-Body Anonymization](https://mlsanity.com/paper/2211.09454/) (2022)
- [Live Face De-Identification in Video](https://mlsanity.com/paper/1911.08348/) (2019)

## Built on by (most cited first, 26 of 26)

- [Edit Away and My Face Will not Stay: Personal Biometric Defense against Malicious Generative Editing](https://mlsanity.com/paper/2411.16832/) (2024)
- [Face De-identification: State-of-the-art Methods and Comparative Studies](https://mlsanity.com/paper/2411.09863/) (2024)
- [Model Will Tell: Training Membership Inference for Diffusion Models](https://mlsanity.com/paper/2403.08487/) (2024)
- [NullFace: Training-Free Localized Face Anonymization](https://mlsanity.com/paper/2503.08478/) (2025)
- [Concept-as-Tree: A Controllable Synthetic Data Framework Makes Stronger Personalized VLMs](https://mlsanity.com/paper/2503.12999/) (2025)
- [D3A-TS: Denoising-Driven Data Augmentation in Time Series](https://mlsanity.com/paper/2312.05550/) (2023)
- [CipherDM: Secure Three-Party Inference for Diffusion Model Sampling](https://mlsanity.com/paper/2409.05414/) (2024)
- [VideoGuard: Protecting Video Content from Unauthorized Editing](https://mlsanity.com/paper/2508.03480/) (2025)
- [UVCG: Leveraging Temporal Consistency for Universal Video Protection](https://mlsanity.com/paper/2411.17746/) (2024)
- [Adversarial Attacks for Good: A Survey of Proactive Protection across the Visual Content Lifecycle](https://mlsanity.com/paper/2608.04314/) (2026)
- [iFADIT: Invertible Face Anonymization via Disentangled Identity Transform](https://mlsanity.com/paper/2501.04390/) (2025)
- [Beyond Inference Intervention: Identity-Decoupled Diffusion for Face Anonymization](https://mlsanity.com/paper/2510.24213/) (2025)
- [Machine Pareidolia: Protecting Facial Image with Emotional Editing](https://mlsanity.com/paper/2603.03665/) (2026)
- [FDeID-Toolbox: Face De-Identification Toolbox](https://mlsanity.com/paper/2603.13121/) (2026)
- [Disentangle Before Anonymize: A Two-stage Framework for Attribute-preserved and Occlusion-robust De-identification](https://mlsanity.com/paper/2311.08786/) (2023)
- [Palmprint De-Identification Using Diffusion Model for High-Quality and Diverse Synthesis](https://mlsanity.com/paper/2504.08272/) (2025)
- [Beyond Anonymization: Object Scrubbing for Privacy-Preserving 2D and 3D Vision Tasks](https://mlsanity.com/paper/2504.16557/) (2025)
- [Controllable Localized Face Anonymization Via Diffusion Inpainting](https://mlsanity.com/paper/2509.14866/) (2025)
- [FLUID: Training-Free Face De-identification via Latent Identity Substitution](https://mlsanity.com/paper/2511.17005/) (2025)
- [SIDeR: Semantic Identity Decoupling for Unrestricted Face Privacy](https://mlsanity.com/paper/2602.04994/) (2026)
- [Face anonymization preserving facial expressions and photometric realism](https://mlsanity.com/paper/2603.17567/) (2026)
- [MD-Face: MoE-Enhanced Label-Free Disentangled Representation for Interactive Facial Attribute Editing](https://mlsanity.com/paper/2604.20317/) (2026)
- [Face De-Identification: A Domain-Centric Survey from Capture to Processing](https://mlsanity.com/paper/2607.25926/) (2026)
- [From Noise to Meaning: Meaningful Secret Sharing with Tamper Detection for Facial Recognition](https://mlsanity.com/paper/2608.08924/) (2026)
- [HYDRO: Towards Non-Reversible Face De-Identification Using a High-Fidelity Hybrid Diffusion and Target-Oriented Approach](https://mlsanity.com/paper/2609.27011/) (2026)
- [No Way To Steal My Face: Proactive Defense Against Identity-Preserving Personalized Generation](https://mlsanity.com/paper/2610.19717/) (2026)

## Abstract (arXiv)

Privacy protection has become a top priority as the proliferation of AI techniques has led to widespread collection and misuse of personal data. Anonymization and visual identity information hiding are two important facial privacy protection tasks that aim to remove identification characteristics from facial images at the human perception level. However, they have a significant difference in that the former aims to prevent the machine from recognizing correctly, while the latter needs to ensure the accuracy of machine recognition. Therefore, it is difficult to train a model to complete these two tasks simultaneously. In this paper, we unify the task of anonymization and visual identity information hiding and propose a novel face privacy protection method based on diffusion models, dubbed Diff-Privacy. Specifically, we train our proposed multi-scale image inversion module (MSI) to obtain a set of SDM format conditional embeddings of the original image. Based on the conditional embeddings, we design corresponding embedding scheduling strategies and construct different energy functions during the denoising process to achieve anonymization and visual identity information hiding. Extensive experiments have been conducted to validate the effectiveness of our proposed framework in protecting facial privacy.