# Disrupting Deepfakes: Adversarial Attacks Against Conditional Image Translation Networks and Facial Manipulation Systems

arXiv: https://arxiv.org/abs/2003.01279 · HTML: https://mlsanity.com/paper/2003.01279/

## Key facts

- Cited by 65 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Mar 2020.

## Built on by (most cited first, 30 of 65)

- [Raising the Cost of Malicious AI-Powered Image Editing](https://mlsanity.com/paper/2302.06588/) (2023)
- [Anti-DreamBooth: Protecting users from personalized text-to-image synthesis](https://mlsanity.com/paper/2303.15433/) (2023)
- [Protecting Facial Privacy: Generating Adversarial Identity Masks via Style-robust Makeup Transfer](https://mlsanity.com/paper/2203.03121/) (2022)
- [CMUA-Watermark: A Cross-Model Universal Adversarial Watermark for Combating Deepfakes](https://mlsanity.com/paper/2105.10872/) (2021)
- [SepMark: Deep Separable Watermarking for Unified Source Tracing and Deepfake Detection](https://mlsanity.com/paper/2305.06321/) (2023)
- [Copyright Protection in Generative AI: A Technical Perspective](https://mlsanity.com/paper/2402.02333/) (2024)
- [Initiative Defense against Facial Manipulation](https://mlsanity.com/paper/2112.10098/) (2021)
- [Proactive Image Manipulation Detection](https://mlsanity.com/paper/2203.15880/) (2022)
- [TAFIM: Targeted Adversarial Attacks against Facial Image Manipulations](https://mlsanity.com/paper/2112.09151/) (2021)
- [MaLP: Manipulation Localization Using a Proactive Scheme](https://mlsanity.com/paper/2303.16976/) (2023)
- [Restricted Black-box Adversarial Attack Against DeepFake Face Swapping](https://mlsanity.com/paper/2204.12347/) (2022)
- [LampMark: Proactive Deepfake Detection via Training-Free Landmark Perceptual Watermarks](https://mlsanity.com/paper/2411.17209/) (2024)
- [Watermark Vaccine: Adversarial Attacks to Prevent Watermark Removal](https://mlsanity.com/paper/2207.08178/) (2022)
- [MagDR: Mask-guided Detection and Reconstruction for Defending Deepfakes](https://mlsanity.com/paper/2103.14211/) (2021)
- [Robust Identity Perceptual Watermark Against Deepfake Face Swapping](https://mlsanity.com/paper/2311.01357/) (2023)
- [ProMark: Proactive Diffusion Watermarking for Causal Attribution](https://mlsanity.com/paper/2403.09914/) (2024)
- [Deepfake Forensics via An Adversarial Game](https://mlsanity.com/paper/2103.13567/) (2021)
- [Attack as the Best Defense: Nullifying Image-to-image Translation GANs via Limit-aware Adversarial Attack](https://mlsanity.com/paper/2110.02516/) (2021)
- [UnGANable: Defending Against GAN-based Face Manipulation](https://mlsanity.com/paper/2210.00957/) (2022)
- [Copyright Protection and Accountability of Generative AI:Attack, Watermarking and Attribution](https://mlsanity.com/paper/2303.09272/) (2023)
- [A Survey of Defenses Against AI-Generated Visual Media: Detection,Disruption, and Authentication](https://mlsanity.com/paper/2407.10575/) (2024)
- [Simulated Adversarial Testing of Face Recognition Models](https://mlsanity.com/paper/2106.04569/) (2021)
- [PoseExaminer: Automated Testing of Out-of-Distribution Robustness in Human Pose and Shape Estimation](https://mlsanity.com/paper/2303.07337/) (2023)
- [DiffVax: Optimization-Free Image Immunization Against Diffusion-Based Editing](https://mlsanity.com/paper/2411.17957/) (2024)
- [Information-containing Adversarial Perturbation for Combating Facial Manipulation Systems](https://mlsanity.com/paper/2303.11625/) (2023)
- [Disrupting Diffusion-based Inpainters with Semantic Digression](https://mlsanity.com/paper/2407.10277/) (2024)
- [FaceShield: Defending Facial Image against Deepfake Threats](https://mlsanity.com/paper/2412.09921/) (2024)
- [NullSwap: Proactive Identity Cloaking Against Deepfake Face Swapping](https://mlsanity.com/paper/2503.18678/) (2025)
- [Silence is Golden: Leveraging Adversarial Examples to Nullify Audio Control in LDM-based Talking-Head Generation](https://mlsanity.com/paper/2506.01591/) (2025)
- [PrObeD: Proactive Object Detection Wrapper](https://mlsanity.com/paper/2310.18788/) (2023)

## Abstract (arXiv)

Face modification systems using deep learning have become increasingly powerful and accessible. Given images of a person's face, such systems can generate new images of that same person under different expressions and poses. Some systems can also modify targeted attributes such as hair color or age. This type of manipulated images and video have been coined Deepfakes. In order to prevent a malicious user from generating modified images of a person without their consent we tackle the new problem of generating adversarial attacks against such image translation systems, which disrupt the resulting output image. We call this problem disrupting deepfakes. Most image translation architectures are generative models conditioned on an attribute (e.g. put a smile on this person's face). We are first to propose and successfully apply (1) class transferable adversarial attacks that generalize to different classes, which means that the attacker does not need to have knowledge about the conditioning class, and (2) adversarial training for generative adversarial networks (GANs) as a first step towards robust image translation networks. Finally, in gray-box scenarios, blurring can mount a successful defense against disruption. We present a spread-spectrum adversarial attack, which evades blur defenses. Our open-source code can be found at https://github.com/natanielruiz/disrupting-deepfakes.