# FakeOut: Leveraging Out-of-domain Self-supervision for Multi-modal Video Deepfake Detection

arXiv: https://arxiv.org/abs/2212.00773 · HTML: https://mlsanity.com/paper/2212.00773/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 50 of its 68 references that are in the map.
- First posted on arXiv 1 Dec 2022.

## Builds on (most cited first, 30 of 50)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [FaceForensics++: Learning to Detect Manipulated Facial Images](https://mlsanity.com/paper/1901.08971/) (2019)
- [HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips](https://mlsanity.com/paper/1906.03327/) (2019)
- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [VoxCeleb2: Deep Speaker Recognition](https://mlsanity.com/paper/1806.05622/) (2018)
- [TSM: Temporal Shift Module for Efficient Video Understanding](https://mlsanity.com/paper/1811.08383/) (2018)
- [CNN-generated images are surprisingly easy to spot... for now](https://mlsanity.com/paper/1912.11035/) (2019)
- [A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild](https://mlsanity.com/paper/2008.10010/) (2020)
- [Face2Face: Real-time Face Capture and Reenactment of RGB Videos](https://mlsanity.com/paper/2007.14808/) (2020)
- [MediaPipe: A Framework for Building Perception Pipelines](https://mlsanity.com/paper/1906.08172/) (2019)
- [Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensics](https://mlsanity.com/paper/1909.12962/) (2019)
- [End-to-End Learning of Visual Representations from Uncurated Instructional Videos](https://mlsanity.com/paper/1912.06430/) (2019)
- [Face X-ray for More General Face Forgery Detection](https://mlsanity.com/paper/1912.13458/) (2019)
- [Look, Listen and Learn](https://mlsanity.com/paper/1705.08168/) (2017)
- [Domain Generalization by Solving Jigsaw Puzzles](https://mlsanity.com/paper/1903.06864/) (2019)
- [FSGAN: Subject Agnostic Face Swapping and Reenactment](https://mlsanity.com/paper/1908.05932/) (2019)
- [ActionCLIP: A New Paradigm for Video Action Recognition](https://mlsanity.com/paper/2109.08472/) (2021)
- [Exposing DeepFake Videos By Detecting Face Warping Artifacts](https://mlsanity.com/paper/1811.00656/) (2018)
- [Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detection](https://mlsanity.com/paper/2012.07657/) (2020)
- [Learning Audio-Visual Speech Representation by Masked Multimodal Cluster Prediction](https://mlsanity.com/paper/2201.02184/) (2022)
- [Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis](https://mlsanity.com/paper/1806.04558/) (2018)
- [Self-Supervised MultiModal Versatile Networks](https://mlsanity.com/paper/2006.16228/) (2020)
- [What makes fake images detectable? Understanding properties that generalize](https://mlsanity.com/paper/2008.10588/) (2020)
- [On the Detection of Digital Face Manipulation](https://mlsanity.com/paper/1910.01717/) (2019)
- [DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detection](https://mlsanity.com/paper/2001.03024/) (2020)
- [Revisiting Self-Supervised Visual Representation Learning](https://mlsanity.com/paper/1901.09005/) (2019)

## Built on by (most cited first, 8 of 8)

- [Unmasking Deepfakes: Masked Autoencoding Spatiotemporal Transformers for Enhanced Video Forgery Detection](https://mlsanity.com/paper/2306.06881/) (2023)
- [Express4D: Expressive, Friendly, and Extensible 4D Facial Motion Generation Benchmark](https://mlsanity.com/paper/2508.12438/) (2025)
- [REED-VAE: RE-Encode Decode Training for Iterative Image Editing with Diffusion Models](https://mlsanity.com/paper/2504.18989/) (2025)
- [Unmasking Puppeteers: Leveraging Biometric Leakage to Expose Impersonation in AI-Based Videoconferencing](https://mlsanity.com/paper/2510.03548/) (2025)
- [V-LASIK: Consistent Glasses-Removal from Videos Using Synthetic Data](https://mlsanity.com/paper/2406.14510/) (2024)
- [FauForensics: Boosting Audio-Visual Deepfake Detection with Facial Action Units](https://mlsanity.com/paper/2505.08294/) (2025)
- [Investigating self-supervised representations for audio-visual deepfake detection](https://mlsanity.com/paper/2511.17181/) (2025)
- [HyperFake: Hyperspectral Reconstruction and Attention-Guided Analysis for Advanced Deepfake Detection](https://mlsanity.com/paper/2505.18587/) (2025)

## Abstract (arXiv)

Video synthesis methods rapidly improved in recent years, allowing easy creation of synthetic humans. This poses a problem, especially in the era of social media, as synthetic videos of speaking humans can be used to spread misinformation in a convincing manner. Thus, there is a pressing need for accurate and robust deepfake detection methods, that can detect forgery techniques not seen during training. In this work, we explore whether this can be done by leveraging a multi-modal, out-of-domain backbone trained in a self-supervised manner, adapted to the video deepfake domain. We propose FakeOut; a novel approach that relies on multi-modal data throughout both the pre-training phase and the adaption phase. We demonstrate the efficacy and robustness of FakeOut in detecting various types of deepfakes, especially manipulations which were not seen during training. Our method achieves state-of-the-art results in cross-dataset generalization on audio-visual datasets. This study shows that, perhaps surprisingly, training on out-of-domain videos (i.e., not especially featuring speaking humans), can lead to better deepfake detection systems. Code is available on GitHub.