# Deepfake Detection via Joint Unsupervised Reconstruction and Supervised Classification

arXiv: https://arxiv.org/abs/2211.13424 · HTML: https://mlsanity.com/paper/2211.13424/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 38 references that are in the map.
- First posted on arXiv 24 Nov 2022.

## Builds on (most cited first, 30 of 30)

- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [FaceForensics++: Learning to Detect Manipulated Facial Images](https://mlsanity.com/paper/1901.08971/) (2019)
- [Face2Face: Real-time Face Capture and Reenactment of RGB Videos](https://mlsanity.com/paper/2007.14808/) (2020)
- [Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensics](https://mlsanity.com/paper/1909.12962/) (2019)
- [Face X-ray for More General Face Forgery Detection](https://mlsanity.com/paper/1912.13458/) (2019)
- [MesoNet: a Compact Facial Video Forgery Detection Network](https://mlsanity.com/paper/1809.00888/) (2018)
- [Multi-attentional Deepfake Detection](https://mlsanity.com/paper/2103.02406/) (2021)
- [Exposing DeepFake Videos By Detecting Face Warping Artifacts](https://mlsanity.com/paper/1811.00656/) (2018)
- [Exposing Deep Fakes Using Inconsistent Head Poses](https://mlsanity.com/paper/1811.00661/) (2018)
- [Deep Video Portraits](https://mlsanity.com/paper/1805.11714/) (2018)
- [On the Detection of Digital Face Manipulation](https://mlsanity.com/paper/1910.01717/) (2019)
- [Deferred Neural Rendering: Image Synthesis using Neural Textures](https://mlsanity.com/paper/1904.12356/) (2019)
- [Capsule-Forensics: Using Capsule Networks to Detect Forged Images and Videos](https://mlsanity.com/paper/1810.11215/) (2018)
- [FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals](https://mlsanity.com/paper/1901.02212/) (2019)
- [Frequency-aware Discriminative Feature Learning Supervised by Single-Center Loss for Face Forgery Detection](https://mlsanity.com/paper/2103.09096/) (2021)
- [Two-Stream Neural Networks for Tampered Face Detection](https://mlsanity.com/paper/1803.11276/) (2018)
- [Fast Face-swap Using Convolutional Neural Networks](https://mlsanity.com/paper/1611.09577/) (2016)
- [Context-Aware Mixup for Domain Adaptive Semantic Segmentation](https://mlsanity.com/paper/2108.03557/) (2021)
- [In Ictu Oculi: Exposing AI Generated Fake Face Videos by Detecting Eye Blinking](https://mlsanity.com/paper/1806.02877/) (2018)
- [Detect and Locate: Exposing Face Manipulation by Semantic- and Noise-level Telltales](https://mlsanity.com/paper/2107.05821/) (2021)
- [PIT: Position-Invariant Transform for Cross-FoV Domain Adaptation](https://mlsanity.com/paper/2108.07142/) (2021)
- [Inverting face embeddings with convolutional neural networks](https://mlsanity.com/paper/1606.04189/) (2016)
- [DeepfakeUCL: Deepfake Detection via Unsupervised Contrastive Learning](https://mlsanity.com/paper/2104.11507/) (2021)
- [Finding Facial Forgery Artifacts with Parts-Based Detectors](https://mlsanity.com/paper/2109.10688/) (2021)
- [High Fidelity Face Manipulation with Extreme Poses and Expressions](https://mlsanity.com/paper/1903.12003/) (2019)
- [Deep Detection for Face Manipulation](https://mlsanity.com/paper/2009.05934/) (2020)
- [Age-Oriented Face Synthesis with Conditional Discriminator Pool and Adversarial Triplet Loss](https://mlsanity.com/paper/2007.00792/) (2020)
- [Explicit Facial Expression Transfer via Fine-Grained Representations](https://mlsanity.com/paper/1909.02967/) (2019)

## Built on by (most cited first, 1 of 1)

- [Revisiting Face Forgery Detection: From Facial Representation to Forgery Detection](https://mlsanity.com/paper/2409.16945/) (2024)

## Abstract (arXiv)

Deep learning has enabled realistic face manipulation (i.e., deepfake), which poses significant concerns over the integrity of the media in circulation. Most existing deep learning techniques for deepfake detection can achieve promising performance in the intra-dataset evaluation setting (i.e., training and testing on the same dataset), but are unable to perform satisfactorily in the inter-dataset evaluation setting (i.e., training on one dataset and testing on another). Most of the previous methods use the backbone network to extract global features for making predictions and only employ binary supervision (i.e., indicating whether the training instances are fake or authentic) to train the network. Classification merely based on the learning of global features leads often leads to weak generalizability to unseen manipulation methods. In addition, the reconstruction task can improve the learned representations. In this paper, we introduce a novel approach for deepfake detection, which considers the reconstruction and classification tasks simultaneously to address these problems. This method shares the information learned by one task with the other, which focuses on a different aspect other existing works rarely consider and hence boosts the overall performance. In particular, we design a two-branch Convolutional AutoEncoder (CAE), in which the Convolutional Encoder used to compress the feature map into the latent representation is shared by both branches. Then the latent representation of the input data is fed to a simple classifier and the unsupervised reconstruction component simultaneously. Our network is trained end-to-end. Experiments demonstrate that our method achieves state-of-the-art performance on three commonly-used datasets, particularly in the cross-dataset evaluation setting.