# Towards Generalizable Deepfake Detection with Locality-aware AutoEncoder

arXiv: https://arxiv.org/abs/1909.05999 · HTML: https://mlsanity.com/paper/1909.05999/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Sep 2019.

## Built on by (most cited first, 17 of 17)

- [Generalizing Face Forgery Detection with High-frequency Features](https://mlsanity.com/paper/2103.12376/) (2021)
- [Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domain](https://mlsanity.com/paper/2103.01856/) (2021)
- [Exploring Temporal Coherence for More General Video Face Forgery Detection](https://mlsanity.com/paper/2108.06693/) (2021)
- [Are GAN generated images easy to detect? A critical analysis of the state-of-the-art](https://mlsanity.com/paper/2104.02617/) (2021)
- [Leveraging Real Talking Faces via Self-Supervision for Robust Forgery Detection](https://mlsanity.com/paper/2201.07131/) (2022)
- [Implicit Identity Leakage: The Stumbling Block to Improving Deepfake Detection Generalization](https://mlsanity.com/paper/2210.14457/) (2022)
- [Face Forensics in the Wild](https://mlsanity.com/paper/2103.16076/) (2021)
- [Countering Malicious DeepFakes: Survey, Battleground, and Horizon](https://mlsanity.com/paper/2103.00218/) (2021)
- [Deepfake Forensics via An Adversarial Game](https://mlsanity.com/paper/2103.13567/) (2021)
- [Improving Synthetically Generated Image Detection in Cross-Concept Settings](https://mlsanity.com/paper/2304.12053/) (2023)
- [Learning Pairwise Interaction for Generalizable DeepFake Detection](https://mlsanity.com/paper/2302.13288/) (2023)
- [Learning Disentangled Representations for Time Series](https://mlsanity.com/paper/2105.08179/) (2021)
- [Towards Generalizable and Robust Face Manipulation Detection via Bag-of-local-feature](https://mlsanity.com/paper/2103.07915/) (2021)
- [Generalized Facial Manipulation Detection with Edge Region Feature Extraction](https://mlsanity.com/paper/2102.01381/) (2021)
- [Deep Insights of Deepfake Technology : A Review](https://mlsanity.com/paper/2105.00192/) (2021)
- [Automated Deepfake Detection](https://mlsanity.com/paper/2106.10705/) (2021)
- [DF-Captcha: A Deepfake Captcha for Preventing Fake Calls](https://mlsanity.com/paper/2208.08524/) (2022)

## Abstract (arXiv)

With advancements of deep learning techniques, it is now possible to generate super-realistic images and videos, i.e., deepfakes. These deepfakes could reach mass audience and result in adverse impacts on our society. Although lots of efforts have been devoted to detect deepfakes, their performance drops significantly on previously unseen but related manipulations and the detection generalization capability remains a problem. Motivated by the fine-grained nature and spatial locality characteristics of deepfakes, we propose Locality-Aware AutoEncoder (LAE) to bridge the generalization gap. In the training process, we use a pixel-wise mask to regularize local interpretation of LAE to enforce the model to learn intrinsic representation from the forgery region, instead of capturing artifacts in the training set and learning superficial correlations to perform detection. We further propose an active learning framework to select the challenging candidates for labeling, which requires human masks for less than 3% of the training data, dramatically reducing the annotation efforts to regularize interpretations. Experimental results on three deepfake detection tasks indicate that LAE could focus on the forgery regions to make decisions. The analysis further shows that LAE outperforms the state-of-the-arts by 6.52%, 12.03%, and 3.08% respectively on three deepfake detection tasks in terms of generalization accuracy on previously unseen manipulations.