# DiffAttack: Evasion Attacks Against Face Recognition via Latent Diffusion Models

arXiv: https://arxiv.org/abs/2607.28936 · HTML: https://mlsanity.com/paper/2607.28936/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Robustness, privacy and adversarial aspects, one of 14 directions in Diffusion models.
- Builds on 34 of its 46 references that are in the map.
- First posted on arXiv 31 Jul 2026.

## Placed in

- Diffusion models / [Robustness, privacy and adversarial aspects](https://mlsanity.com/diffusion/robustness-privacy-and-adversarial-aspects/)

## Builds on (most cited first, 30 of 34)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Multi-Concept Customization of Text-to-Image Diffusion](https://mlsanity.com/paper/2212.04488/) (2022)
- [Diffusion Models for Adversarial Purification](https://mlsanity.com/paper/2205.07460/) (2022)
- [Evading Defenses to Transferable Adversarial Examples by Translation-Invariant Attacks](https://mlsanity.com/paper/1904.02884/) (2019)
- [AdvHat: Real-world adversarial attack on ArcFace Face ID system](https://mlsanity.com/paper/1908.08705/) (2019)
- [Adv-Makeup: A New Imperceptible and Transferable Attack on Face Recognition](https://mlsanity.com/paper/2105.03162/) (2021)
- [Protecting Facial Privacy: Generating Adversarial Identity Masks via Style-robust Makeup Transfer](https://mlsanity.com/paper/2203.03121/) (2022)
- [Towards Face Encryption by Generating Adversarial Identity Masks](https://mlsanity.com/paper/2003.06814/) (2020)
- [From Facial Expression Recognition to Interpersonal Relation Prediction](https://mlsanity.com/paper/1609.06426/) (2016)
- [Towards Privacy-Preserving Visual Recognition via Adversarial Training: A Pilot Study](https://mlsanity.com/paper/1807.08379/) (2018)
- [Adv-Attribute: Inconspicuous and Transferable Adversarial Attack on Face Recognition](https://mlsanity.com/paper/2210.06871/) (2022)
- [CLIP2Protect: Protecting Facial Privacy using Text-Guided Makeup via Adversarial Latent Search](https://mlsanity.com/paper/2306.10008/) (2023)
- [FakeLocator: Robust Localization of GAN-Based Face Manipulations](https://mlsanity.com/paper/2001.09598/) (2020)
- [DiffProtect: Generate Adversarial Examples with Diffusion Models for Facial Privacy Protection](https://mlsanity.com/paper/2305.13625/) (2023)
- [DiffAM: Diffusion-based Adversarial Makeup Transfer for Facial Privacy Protection](https://mlsanity.com/paper/2405.09882/) (2024)
- [PrivacyNet: Semi-Adversarial Networks for Multi-attribute Face Privacy](https://mlsanity.com/paper/2001.00561/) (2020)
- [Privacy-preserving Adversarial Facial Features](https://mlsanity.com/paper/2305.05391/) (2023)
- [Privacy-Preserving Face Recognition Using Trainable Feature Subtraction](https://mlsanity.com/paper/2403.12457/) (2024)
- [Adv-Diffusion: Imperceptible Adversarial Face Identity Attack via Latent Diffusion Model](https://mlsanity.com/paper/2312.11285/) (2023)
- [PFA-GAN: Progressive Face Aging with Generative Adversarial Network](https://mlsanity.com/paper/2012.03459/) (2020)
- [SoK: Anti-Facial Recognition Technology](https://mlsanity.com/paper/2112.04558/) (2021)
- [GAN-based Facial Attribute Manipulation](https://mlsanity.com/paper/2210.12683/) (2022)
- [Adv-CPG: A Customized Portrait Generation Framework with Facial Adversarial Attacks](https://mlsanity.com/paper/2503.08269/) (2025)
- [Personalized Privacy Protection Mask Against Unauthorized Facial Recognition](https://mlsanity.com/paper/2407.13975/) (2024)

## Abstract (arXiv)

Facial biometric identification relies on the distinctiveness of user attributes within a high-dimensional embedding space. However, the decision boundaries of deep face recognition (FR) systems are often sufficiently narrow that they can be conflated, rendering the models vulnerable to adversarial attacks. In such scenarios, the FR system fails to distinguish between an authentic source and a meticulously crafted adversarial face. Existing adversarial methods targeting facial biometrics are limited in both performance and their ability to generate high-quality images that are imperceptible to humans. Moreover, these methods often fail when the source and target images belong to different demographic groups or genders. To address these limitations, we present a novel approach for adversarial face generation via latent-space optimization. We leverage latent diffusion models directly to guide generation toward target identity embeddings, as measured by a face recognition model. Our proposed DiffAttack framework has been evaluated on standard benchmarks, such as the FFHQ and CelebA-HQ datasets. DiffAttack significantly outperforms existing adversarial techniques, achieving a high average attack success rate of 84.86% across multiple face recognition models (e.g., FaceNet). Notably, DiffAttack demonstrates superior transferability, surpassing traditional noise-based methods by over 15.28% and semantic-based approaches by approximately 5.21% on benchmark datasets like FFHQ and CelebA-HQ.