# Exploring Adversarial Fake Images on Face Manifold

arXiv: https://arxiv.org/abs/2101.03272 · HTML: https://mlsanity.com/paper/2101.03272/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 43 references that are in the map.
- First posted on arXiv 9 Jan 2021.

## Builds on (most cited first, 30 of 32)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Deep Learning Face Attributes in the Wild](https://mlsanity.com/paper/1411.7766/) (2014)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [FaceForensics++: Learning to Detect Manipulated Facial Images](https://mlsanity.com/paper/1901.08971/) (2019)
- [Distillation as a Defense to Adversarial Perturbations against Deep Neural Networks](https://mlsanity.com/paper/1511.04508/) (2015)
- [Face2Face: Real-time Face Capture and Reenactment of RGB Videos](https://mlsanity.com/paper/2007.14808/) (2020)
- [Delving into Transferable Adversarial Examples and Black-box Attacks](https://mlsanity.com/paper/1611.02770/) (2016)
- [Face X-ray for More General Face Forgery Detection](https://mlsanity.com/paper/1912.13458/) (2019)
- [Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples](https://mlsanity.com/paper/1605.07277/) (2016)
- [MesoNet: a Compact Facial Video Forgery Detection Network](https://mlsanity.com/paper/1809.00888/) (2018)
- [Defense against Adversarial Attacks Using High-Level Representation Guided Denoiser](https://mlsanity.com/paper/1712.02976/) (2017)
- [AttGAN: Facial Attribute Editing by Only Changing What You Want](https://mlsanity.com/paper/1711.10678/) (2017)
- [DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detection](https://mlsanity.com/paper/2001.00179/) (2020)
- [Spatially Transformed Adversarial Examples](https://mlsanity.com/paper/1801.02612/) (2018)
- [Constructing Unrestricted Adversarial Examples with Generative Models](https://mlsanity.com/paper/1805.07894/) (2018)
- [Two-Stream Neural Networks for Tampered Face Detection](https://mlsanity.com/paper/1803.11276/) (2018)
- [Fast Face-swap Using Convolutional Neural Networks](https://mlsanity.com/paper/1611.09577/) (2016)
- [Decoupling Direction and Norm for Efficient Gradient-Based L2 Adversarial Attacks and Defenses](https://mlsanity.com/paper/1811.09600/) (2018)
- [SemanticAdv: Generating Adversarial Examples via Attribute-conditional Image Editing](https://mlsanity.com/paper/1906.07927/) (2019)
- [Exploring the Landscape of Spatial Robustness](https://mlsanity.com/paper/1712.02779/) (2017)
- [Automatic Face Reenactment](https://mlsanity.com/paper/1602.02651/) (2016)
- [AT-GAN: An Adversarial Generator Model for Non-constrained Adversarial Examples](https://mlsanity.com/paper/1904.07793/) (2019)

## Built on by (most cited first, 24 of 24)

- [Advances in adversarial attacks and defenses in computer vision: A survey](https://mlsanity.com/paper/2108.00401/) (2021)
- [Exploring Frequency Adversarial Attacks for Face Forgery Detection](https://mlsanity.com/paper/2203.15674/) (2022)
- [Evading DeepFake Detectors via Adversarial Statistical Consistency](https://mlsanity.com/paper/2304.11670/) (2023)
- [DISCO: Adversarial Defense with Local Implicit Functions](https://mlsanity.com/paper/2212.05630/) (2022)
- [An Analysis of Recent Advances in Deepfake Image Detection in an Evolving Threat Landscape](https://mlsanity.com/paper/2404.16212/) (2024)
- [Are Watermarks Bugs for Deepfake Detectors? Rethinking Proactive Forensics](https://mlsanity.com/paper/2404.17867/) (2024)
- [A Survey of Defenses Against AI-Generated Visual Media: Detection,Disruption, and Authentication](https://mlsanity.com/paper/2407.10575/) (2024)
- [AVA: Inconspicuous Attribute Variation-based Adversarial Attack bypassing DeepFake Detection](https://mlsanity.com/paper/2312.08675/) (2023)
- [Vulnerabilities in AI-generated Image Detection: The Challenge of Adversarial Attacks](https://mlsanity.com/paper/2407.20836/) (2024)
- [Adversarial Robustness of AI-Generated Image Detectors in the Real World](https://mlsanity.com/paper/2410.01574/) (2024)
- [Evading Forensic Classifiers with Attribute-Conditioned Adversarial Faces](https://mlsanity.com/paper/2306.13091/) (2023)
- [Transferable Adversarial Face Attack with Text Controlled Attribute](https://mlsanity.com/paper/2412.11735/) (2024)
- [Unsupervised Model Diagnosis](https://mlsanity.com/paper/2410.06243/) (2024)
- [Passive Deepfake Detection Across Multi-modalities: A Comprehensive Survey](https://mlsanity.com/paper/2411.17911/) (2024)
- [Feature Extraction Matters More: Universal Deepfake Disruption through Attacking Ensemble Feature Extractors](https://mlsanity.com/paper/2303.00200/) (2023)
- [Turn Fake into Real: Adversarial Head Turn Attacks Against Deepfake Detection](https://mlsanity.com/paper/2309.01104/) (2023)
- [Robust Deepfake On Unrestricted Media: Generation And Detection](https://mlsanity.com/paper/2202.06228/) (2022)
- [Principles of Designing Robust Remote Face Anti-Spoofing Systems](https://mlsanity.com/paper/2406.03684/) (2024)
- [Semantic Image Attack for Visual Model Diagnosis](https://mlsanity.com/paper/2303.13010/) (2023)
- [Towards Assessing and Characterizing the Semantic Robustness of Face Recognition](https://mlsanity.com/paper/2202.04978/) (2022)
- [Exploring the Adversarial Robustness of Face Forgery Detection with Decision-based Black-box Attacks](https://mlsanity.com/paper/2310.12017/) (2023)
- [Counterfactual Explanations for Face Forgery Detection via Adversarial Removal of Artifacts](https://mlsanity.com/paper/2404.08341/) (2024)
- [Current Status and Trends in Image Anti-Forensics Research: A Bibliometric Analysis](https://mlsanity.com/paper/2408.11365/) (2024)
- [Active Fake: DeepFake Camouflage](https://mlsanity.com/paper/2409.03200/) (2024)

## Abstract (arXiv)

Images synthesized by powerful generative adversarial network (GAN) based methods have drawn moral and privacy concerns. Although image forensic models have reached great performance in detecting fake images from real ones, these models can be easily fooled with a simple adversarial attack. But, the noise adding adversarial samples are also arousing suspicion. In this paper, instead of adding adversarial noise, we optimally search adversarial points on face manifold to generate anti-forensic fake face images. We iteratively do a gradient-descent with each small step in the latent space of a generative model, e.g. Style-GAN, to find an adversarial latent vector, which is similar to norm-based adversarial attack but in latent space. Then, the generated fake images driven by the adversarial latent vectors with the help of GANs can defeat main-stream forensic models. For examples, they make the accuracy of deepfake detection models based on Xception or EfficientNet drop from over 90% to nearly 0%, meanwhile maintaining high visual quality. In addition, we find manipulating style vector z or noise vectors n at different levels have impacts on attack success rate. The generated adversarial images mainly have facial texture or face attributes changing.