# When Handcrafted Features and Deep Features Meet Mismatched Training and Test Sets for Deepfake Detection

arXiv: https://arxiv.org/abs/2209.13289 · HTML: https://mlsanity.com/paper/2209.13289/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 29 of its 49 references that are in the map.
- First posted on arXiv 27 Sep 2022.

## Builds on (most cited first, 29 of 29)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [FaceForensics++: Learning to Detect Manipulated Facial Images](https://mlsanity.com/paper/1901.08971/) (2019)
- [VGGFace2: A dataset for recognising faces across pose and age](https://mlsanity.com/paper/1710.08092/) (2017)
- [CNN-generated images are surprisingly easy to spot... for now](https://mlsanity.com/paper/1912.11035/) (2019)
- [Face2Face: Real-time Face Capture and Reenactment of RGB Videos](https://mlsanity.com/paper/2007.14808/) (2020)
- [Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Clues](https://mlsanity.com/paper/2007.09355/) (2020)
- [Face X-ray for More General Face Forgery Detection](https://mlsanity.com/paper/1912.13458/) (2019)
- [MesoNet: a Compact Facial Video Forgery Detection Network](https://mlsanity.com/paper/1809.00888/) (2018)
- [Exposing DeepFake Videos By Detecting Face Warping Artifacts](https://mlsanity.com/paper/1811.00656/) (2018)
- [Exposing Deep Fakes Using Inconsistent Head Poses](https://mlsanity.com/paper/1811.00661/) (2018)
- [Detecting and Simulating Artifacts in GAN Fake Images](https://mlsanity.com/paper/1907.06515/) (2019)
- [On the Detection of Digital Face Manipulation](https://mlsanity.com/paper/1910.01717/) (2019)
- [Deferred Neural Rendering: Image Synthesis using Neural Textures](https://mlsanity.com/paper/1904.12356/) (2019)
- [Global Texture Enhancement for Fake Face Detection in the Wild](https://mlsanity.com/paper/2002.00133/) (2020)
- [Do GANs leave artificial fingerprints?](https://mlsanity.com/paper/1812.11842/) (2018)
- [Two-Stream Neural Networks for Tampered Face Detection](https://mlsanity.com/paper/1803.11276/) (2018)
- [Recurrent Convolutional Strategies for Face Manipulation Detection in Videos](https://mlsanity.com/paper/1905.00582/) (2019)
- [DeepFake Detection by Analyzing Convolutional Traces](https://mlsanity.com/paper/2004.10448/) (2020)
- [Unmasking DeepFakes with simple Features](https://mlsanity.com/paper/1911.00686/) (2019)
- [Use of a Capsule Network to Detect Fake Images and Videos](https://mlsanity.com/paper/1910.12467/) (2019)
- [Video Face Manipulation Detection Through Ensemble of CNNs](https://mlsanity.com/paper/2004.07676/) (2020)
- [Recasting Residual-based Local Descriptors as Convolutional Neural Networks: an Application to Image Forgery Detection](https://mlsanity.com/paper/1703.04615/) (2017)
- [CNN Detection of GAN-Generated Face Images based on Cross-Band Co-occurrences Analysis](https://mlsanity.com/paper/2007.12909/) (2020)
- [Fighting Deepfake by Exposing the Convolutional Traces on Images](https://mlsanity.com/paper/2008.04095/) (2020)
- [Identification of Deep Network Generated Images Using Disparities in Color Components](https://mlsanity.com/paper/1808.07276/) (2018)
- [Swapped Face Detection using Deep Learning and Subjective Assessment](https://mlsanity.com/paper/1909.04217/) (2019)
- [Complement Face Forensic Detection and Localization with FacialLandmarks](https://mlsanity.com/paper/1910.05455/) (2019)
- [Deep Face Forgery Detection](https://mlsanity.com/paper/2004.11804/) (2020)

## Abstract (arXiv)

The accelerated growth in synthetic visual media generation and manipulation has now reached the point of raising significant concerns and posing enormous intimidations towards society. There is an imperative need for automatic detection networks towards false digital content and avoid the spread of dangerous artificial information to contend with this threat. In this paper, we utilize and compare two kinds of handcrafted features(SIFT and HoG) and two kinds of deep features(Xception and CNN+RNN) for the deepfake detection task. We also check the performance of these features when there are mismatches between training sets and test sets. Evaluation is performed on the famous FaceForensics++ dataset, which contains four sub-datasets, Deepfakes, Face2Face, FaceSwap and NeuralTextures. The best results are from Xception, where the accuracy could surpass over 99% when the training and test set are both from the same sub-dataset. In comparison, the results drop dramatically when the training set mismatches the test set. This phenomenon reveals the challenge of creating a universal deepfake detection system.