# Initiative Defense against Facial Manipulation

arXiv: https://arxiv.org/abs/2112.10098 · HTML: https://mlsanity.com/paper/2112.10098/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 31 references that are in the map.
- First posted on arXiv 19 Dec 2021.

## Builds on (most cited first, 24 of 24)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Perceptual Losses for Real-Time Style Transfer and Super-Resolution](https://mlsanity.com/paper/1603.08155/) (2016)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [FaceForensics++: Learning to Detect Manipulated Facial Images](https://mlsanity.com/paper/1901.08971/) (2019)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentation](https://mlsanity.com/paper/1808.00897/) (2018)
- [Face2Face: Real-time Face Capture and Reenactment of RGB Videos](https://mlsanity.com/paper/2007.14808/) (2020)
- [Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Clues](https://mlsanity.com/paper/2007.09355/) (2020)
- [Face X-ray for More General Face Forgery Detection](https://mlsanity.com/paper/1912.13458/) (2019)
- [MesoNet: a Compact Facial Video Forgery Detection Network](https://mlsanity.com/paper/1809.00888/) (2018)
- [FSGAN: Subject Agnostic Face Swapping and Reenactment](https://mlsanity.com/paper/1908.05932/) (2019)
- [Deep Video Portraits](https://mlsanity.com/paper/1805.11714/) (2018)
- [On the Detection of Digital Face Manipulation](https://mlsanity.com/paper/1910.01717/) (2019)
- [Two-Stream Neural Networks for Tampered Face Detection](https://mlsanity.com/paper/1803.11276/) (2018)
- [Fast Face-swap Using Convolutional Neural Networks](https://mlsanity.com/paper/1611.09577/) (2016)
- [ReenactGAN: Learning to Reenact Faces via Boundary Transfer](https://mlsanity.com/paper/1807.11079/) (2018)
- [Towards Open-Set Identity Preserving Face Synthesis](https://mlsanity.com/paper/1803.11182/) (2018)
- [Learning to Confuse: Generating Training Time Adversarial Data with Auto-Encoder](https://mlsanity.com/paper/1905.09027/) (2019)
- [Disrupting Deepfakes: Adversarial Attacks Against Conditional Image Translation Networks and Facial Manipulation Systems](https://mlsanity.com/paper/2003.01279/) (2020)
- [Use of a Capsule Network to Detect Fake Images and Videos](https://mlsanity.com/paper/1910.12467/) (2019)
- [RSGAN: Face Swapping and Editing using Face and Hair Representation in Latent Spaces](https://mlsanity.com/paper/1804.03447/) (2018)
- [Fawkes: Protecting Privacy against Unauthorized Deep Learning Models](https://mlsanity.com/paper/2002.08327/) (2020)

## Built on by (most cited first, 30 of 36)

- [OPERA: Alleviating Hallucination in Multi-Modal Large Language Models via Over-Trust Penalty and Retrospection-Allocation](https://mlsanity.com/paper/2311.17911/) (2023)
- [Anti-DreamBooth: Protecting users from personalized text-to-image synthesis](https://mlsanity.com/paper/2303.15433/) (2023)
- [SepMark: Deep Separable Watermarking for Unified Source Tracing and Deepfake Detection](https://mlsanity.com/paper/2305.06321/) (2023)
- [Copyright Protection in Generative AI: A Technical Perspective](https://mlsanity.com/paper/2402.02333/) (2024)
- [SimAC: A Simple Anti-Customization Method for Protecting Face Privacy against Text-to-Image Synthesis of Diffusion Models](https://mlsanity.com/paper/2312.07865/) (2023)
- [LampMark: Proactive Deepfake Detection via Training-Free Landmark Perceptual Watermarks](https://mlsanity.com/paper/2411.17209/) (2024)
- [Robust Identity Perceptual Watermark Against Deepfake Face Swapping](https://mlsanity.com/paper/2311.01357/) (2023)
- [UnGANable: Defending Against GAN-based Face Manipulation](https://mlsanity.com/paper/2210.00957/) (2022)
- [A Survey of Defenses Against AI-Generated Visual Media: Detection,Disruption, and Authentication](https://mlsanity.com/paper/2407.10575/) (2024)
- [Dormant: Defending against Pose-driven Human Image Animation](https://mlsanity.com/paper/2409.14424/) (2024)
- [PointCAT: Contrastive Adversarial Training for Robust Point Cloud Recognition](https://mlsanity.com/paper/2209.07788/) (2022)
- [Hierarchical Forgery Classifier On Multi-modality Face Forgery Clues](https://mlsanity.com/paper/2212.14629/) (2022)
- [Information-containing Adversarial Perturbation for Combating Facial Manipulation Systems](https://mlsanity.com/paper/2303.11625/) (2023)
- [Towards Prompt-robust Face Privacy Protection via Adversarial Decoupling Augmentation Framework](https://mlsanity.com/paper/2305.03980/) (2023)
- [Disrupting Diffusion-based Inpainters with Semantic Digression](https://mlsanity.com/paper/2407.10277/) (2024)
- [NullSwap: Proactive Identity Cloaking Against Deepfake Face Swapping](https://mlsanity.com/paper/2503.18678/) (2025)
- [Evolving from Single-modal to Multi-modal Facial Deepfake Detection: Progress and Challenges](https://mlsanity.com/paper/2406.06965/) (2024)
- [PID: Prompt-Independent Data Protection Against Latent Diffusion Models](https://mlsanity.com/paper/2406.15305/) (2024)
- [ID-Guard: A Universal Framework for Combating Facial Manipulation via Breaking Identification](https://mlsanity.com/paper/2409.13349/) (2024)
- [FaceSwapGuard: Safeguarding Facial Privacy from DeepFake Threats through Identity Obfuscation](https://mlsanity.com/paper/2502.10801/) (2025)
- [Feature Extraction Matters More: Universal Deepfake Disruption through Attacking Ensemble Feature Extractors](https://mlsanity.com/paper/2303.00200/) (2023)
- [Big Brother is Watching: Proactive Deepfake Detection via Learnable Hidden Face](https://mlsanity.com/paper/2504.11309/) (2025)
- [LEAT: Towards Robust Deepfake Disruption in Real-World Scenarios via Latent Ensemble Attack](https://mlsanity.com/paper/2307.01520/) (2023)
- [MITS-GAN: Safeguarding Medical Imaging from Tampering with Generative Adversarial Networks](https://mlsanity.com/paper/2401.09624/) (2024)
- [Towards Robust Protective Perturbation against DeepFake Face Swapping](https://mlsanity.com/paper/2512.07228/) (2025)
- [Off-The-Shelf Image-to-Image Models Are All You Need To Defeat Image Protection Schemes](https://mlsanity.com/paper/2602.22197/) (2026)
- [Diffusion-Guided Adversarial Perturbation Injection for Generalizable Defense Against Facial Manipulations](https://mlsanity.com/paper/2604.01635/) (2026)
- [Transferable Attack against Face Swapping in an Extended Space](https://mlsanity.com/paper/2606.25376/) (2026)
- [Searching for the Fakes: Efficient Neural Architecture Search for General Face Forgery Detection](https://mlsanity.com/paper/2306.08830/) (2023)
- [Dual Defense: Adversarial, Traceable, and Invisible Robust Watermarking against Face Swapping](https://mlsanity.com/paper/2310.16540/) (2023)

## Abstract (arXiv)

Benefiting from the development of generative adversarial networks (GAN), facial manipulation has achieved significant progress in both academia and industry recently. It inspires an increasing number of entertainment applications but also incurs severe threats to individual privacy and even political security meanwhile. To mitigate such risks, many countermeasures have been proposed. However, the great majority methods are designed in a passive manner, which is to detect whether the facial images or videos are tampered after their wide propagation. These detection-based methods have a fatal limitation, that is, they only work for ex-post forensics but can not prevent the engendering of malicious behavior. To address the limitation, in this paper, we propose a novel framework of initiative defense to degrade the performance of facial manipulation models controlled by malicious users. The basic idea is to actively inject imperceptible venom into target facial data before manipulation. To this end, we first imitate the target manipulation model with a surrogate model, and then devise a poison perturbation generator to obtain the desired venom. An alternating training strategy are further leveraged to train both the surrogate model and the perturbation generator. Two typical facial manipulation tasks: face attribute editing and face reenactment, are considered in our initiative defense framework. Extensive experiments demonstrate the effectiveness and robustness of our framework in different settings. Finally, we hope this work can shed some light on initiative countermeasures against more adversarial scenarios.