# Protecting Facial Privacy: Generating Adversarial Identity Masks via Style-robust Makeup Transfer

arXiv: https://arxiv.org/abs/2203.03121 · HTML: https://mlsanity.com/paper/2203.03121/

## Key facts

- Cited by 78 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 48 references that are in the map.
- First posted on arXiv 7 Mar 2022.

## Builds on (most cited first, 30 of 31)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks](https://mlsanity.com/paper/1809.00219/) (2018)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks](https://mlsanity.com/paper/1703.10593/) (2017)
- [Improving Transferability of Adversarial Examples with Input Diversity](https://mlsanity.com/paper/1803.06978/) (2018)
- [A Neural Algorithm of Artistic Style](https://mlsanity.com/paper/1508.06576/) (2015)
- [Evading Defenses to Transferable Adversarial Examples by Translation-Invariant Attacks](https://mlsanity.com/paper/1904.02884/) (2019)
- [Nesterov Accelerated Gradient and Scale Invariance for Adversarial Attacks](https://mlsanity.com/paper/1908.06281/) (2019)
- [Generating Adversarial Examples with Adversarial Networks](https://mlsanity.com/paper/1801.02610/) (2018)
- [Efficient Decision-based Black-box Adversarial Attacks on Face Recognition](https://mlsanity.com/paper/1904.04433/) (2019)
- [AdvHat: Real-world adversarial attack on ArcFace Face ID system](https://mlsanity.com/paper/1908.08705/) (2019)
- [Adv-Makeup: A New Imperceptible and Transferable Attack on Face Recognition](https://mlsanity.com/paper/2105.03162/) (2021)
- [LowKey: Leveraging Adversarial Attacks to Protect Social Media Users from Facial Recognition](https://mlsanity.com/paper/2101.07922/) (2021)
- [SemanticAdv: Generating Adversarial Examples via Attribute-conditional Image Editing](https://mlsanity.com/paper/1906.07927/) (2019)
- [Towards Transferable Adversarial Attack against Deep Face Recognition](https://mlsanity.com/paper/2004.05790/) (2020)
- [Improving Transferability of Adversarial Patches on Face Recognition with Generative Models](https://mlsanity.com/paper/2106.15058/) (2021)
- [PSGAN: Pose and Expression Robust Spatial-Aware GAN for Customizable Makeup Transfer](https://mlsanity.com/paper/1909.06956/) (2019)
- [Disrupting Deepfakes: Adversarial Attacks Against Conditional Image Translation Networks and Facial Manipulation Systems](https://mlsanity.com/paper/2003.01279/) (2020)
- [LADN: Local Adversarial Disentangling Network for Facial Makeup and De-Makeup](https://mlsanity.com/paper/1904.11272/) (2019)
- [Towards Face Encryption by Generating Adversarial Identity Masks](https://mlsanity.com/paper/2003.06814/) (2020)
- [Unpaired Image-to-Image Translation using Adversarial Consistency Loss](https://mlsanity.com/paper/2003.04858/) (2020)

## Built on by (most cited first, 30 of 78)

- [Anti-DreamBooth: Protecting users from personalized text-to-image synthesis](https://mlsanity.com/paper/2303.15433/) (2023)
- [AdvCLIP: Downstream-agnostic Adversarial Examples in Multimodal Contrastive Learning](https://mlsanity.com/paper/2308.07026/) (2023)
- [Stable-Makeup: When Real-World Makeup Transfer Meets Diffusion Model](https://mlsanity.com/paper/2403.07764/) (2024)
- [CLIP2Protect: Protecting Facial Privacy using Text-Guided Makeup via Adversarial Latent Search](https://mlsanity.com/paper/2306.10008/) (2023)
- [One Perturbation is Enough: On Generating Universal Adversarial Perturbations against Vision-Language Pre-training Models](https://mlsanity.com/paper/2406.05491/) (2024)
- [DiffProtect: Generate Adversarial Examples with Diffusion Models for Facial Privacy Protection](https://mlsanity.com/paper/2305.13625/) (2023)
- [Why Does Little Robustness Help? A Further Step Towards Understanding Adversarial Transferability](https://mlsanity.com/paper/2307.07873/) (2023)
- [Downstream-agnostic Adversarial Examples](https://mlsanity.com/paper/2307.12280/) (2023)
- [DiffAM: Diffusion-based Adversarial Makeup Transfer for Facial Privacy Protection](https://mlsanity.com/paper/2405.09882/) (2024)
- [Adv-Diffusion: Imperceptible Adversarial Face Identity Attack via Latent Diffusion Model](https://mlsanity.com/paper/2312.11285/) (2023)
- [Securely Fine-tuning Pre-trained Encoders Against Adversarial Examples](https://mlsanity.com/paper/2403.10801/) (2024)
- [Geometry Cloak: Preventing TGS-based 3D Reconstruction from Copyrighted Images](https://mlsanity.com/paper/2410.22705/) (2024)
- [Discrete Point-wise Attack Is Not Enough: Generalized Manifold Adversarial Attack for Face Recognition](https://mlsanity.com/paper/2301.06083/) (2022)
- [Adv-CPG: A Customized Portrait Generation Framework with Facial Adversarial Attacks](https://mlsanity.com/paper/2503.08269/) (2025)
- [Detection and Defense of Unlearnable Examples](https://mlsanity.com/paper/2312.08898/) (2023)
- [StyLess: Boosting the Transferability of Adversarial Examples](https://mlsanity.com/paper/2304.11579/) (2023)
- [Personalized Privacy Protection Mask Against Unauthorized Facial Recognition](https://mlsanity.com/paper/2407.13975/) (2024)
- [PointCA: Evaluating the Robustness of 3D Point Cloud Completion Models Against Adversarial Examples](https://mlsanity.com/paper/2211.12294/) (2022)
- [GANonymization: A GAN-based Face Anonymization Framework for Preserving Emotional Expressions](https://mlsanity.com/paper/2305.02143/) (2023)
- [Towards Prompt-robust Face Privacy Protection via Adversarial Decoupling Augmentation Framework](https://mlsanity.com/paper/2305.03980/) (2023)
- [Face De-identification: State-of-the-art Methods and Comparative Studies](https://mlsanity.com/paper/2411.09863/) (2024)
- [Rethinking Impersonation and Dodging Attacks on Face Recognition Systems](https://mlsanity.com/paper/2401.08903/) (2024)
- [Makeup-Guided Facial Privacy Protection via Untrained Neural Network Priors](https://mlsanity.com/paper/2408.12387/) (2024)
- [Improving the Transferability of Adversarial Attacks on Face Recognition with Beneficial Perturbation Feature Augmentation](https://mlsanity.com/paper/2210.16117/) (2022)
- [Enhancing Facial Privacy Protection via Weakening Diffusion Purification](https://mlsanity.com/paper/2503.10350/) (2025)
- [Consistent Attack: Universal Adversarial Perturbation on Embodied Vision Navigation](https://mlsanity.com/paper/2206.05751/) (2022)
- [Face Encryption via Frequency-Restricted Identity-Agnostic Attacks](https://mlsanity.com/paper/2308.05983/) (2023)
- [Beyond Boundaries: A Comprehensive Survey of Transferable Attacks on AI Systems](https://mlsanity.com/paper/2311.11796/) (2023)
- [A Survey of Privacy-Preserving Model Explanations: Privacy Risks, Attacks, and Countermeasures](https://mlsanity.com/paper/2404.00673/) (2024)
- [Scale-free and Task-agnostic Attack: Generating Photo-realistic Adversarial Patterns with Patch Quilting Generator](https://mlsanity.com/paper/2208.06222/) (2022)

## Abstract (arXiv)

While deep face recognition (FR) systems have shown amazing performance in identification and verification, they also arouse privacy concerns for their excessive surveillance on users, especially for public face images widely spread on social networks. Recently, some studies adopt adversarial examples to protect photos from being identified by unauthorized face recognition systems. However, existing methods of generating adversarial face images suffer from many limitations, such as awkward visual, white-box setting, weak transferability, making them difficult to be applied to protect face privacy in reality. In this paper, we propose adversarial makeup transfer GAN (AMT-GAN), a novel face protection method aiming at constructing adversarial face images that preserve stronger black-box transferability and better visual quality simultaneously. AMT-GAN leverages generative adversarial networks (GAN) to synthesize adversarial face images with makeup transferred from reference images. In particular, we introduce a new regularization module along with a joint training strategy to reconcile the conflicts between the adversarial noises and the cycle consistence loss in makeup transfer, achieving a desirable balance between the attack strength and visual changes. Extensive experiments verify that compared with state of the arts, AMT-GAN can not only preserve a comfortable visual quality, but also achieve a higher attack success rate over commercial FR APIs, including Face++, Aliyun, and Microsoft.