# One Detector to Rule Them All: Towards a General Deepfake Attack Detection Framework

arXiv: https://arxiv.org/abs/2105.00187 · HTML: https://mlsanity.com/paper/2105.00187/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 57 references that are in the map.
- First posted on arXiv 1 May 2021.

## Builds on (most cited first, 27 of 27)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcasting](https://mlsanity.com/paper/1506.04214/) (2015)
- [Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/1702.05464/) (2017)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [FaceForensics++: Learning to Detect Manipulated Facial Images](https://mlsanity.com/paper/1901.08971/) (2019)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.03213/) (2017)
- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [First Order Motion Model for Image Animation](https://mlsanity.com/paper/2003.00196/) (2020)
- [Face2Face: Real-time Face Capture and Reenactment of RGB Videos](https://mlsanity.com/paper/2007.14808/) (2020)
- [Generating Sequences With Recurrent Neural Networks](https://mlsanity.com/paper/1308.0850/) (2013)
- [MesoNet: a Compact Facial Video Forgery Detection Network](https://mlsanity.com/paper/1809.00888/) (2018)
- [Measuring Catastrophic Forgetting in Neural Networks](https://mlsanity.com/paper/1708.02072/) (2017)
- [Few-Shot Adversarial Learning of Realistic Neural Talking Head Models](https://mlsanity.com/paper/1905.08233/) (2019)
- [The Creation and Detection of Deepfakes: A Survey](https://mlsanity.com/paper/2004.11138/) (2020)
- [The Deepfake Detection Challenge (DFDC) Preview Dataset](https://mlsanity.com/paper/1910.08854/) (2019)
- [Learning Rich Features for Image Manipulation Detection](https://mlsanity.com/paper/1805.04953/) (2018)
- [Fighting Fake News: Image Splice Detection via Learned Self-Consistency](https://mlsanity.com/paper/1805.04096/) (2018)
- [Semantic Autoencoder for Zero-Shot Learning](https://mlsanity.com/paper/1704.08345/) (2017)
- [Recurrent Convolutional Strategies for Face Manipulation Detection in Videos](https://mlsanity.com/paper/1905.00582/) (2019)
- [ForensicTransfer: Weakly-supervised Domain Adaptation for Forgery Detection](https://mlsanity.com/paper/1812.02510/) (2018)
- [Image Splicing Localization Using A Multi-Task Fully Convolutional Network (MFCN)](https://mlsanity.com/paper/1709.02016/) (2017)
- [T-GD: Transferable GAN-generated Images Detection Framework](https://mlsanity.com/paper/2008.04115/) (2020)
- [A Convolutional LSTM based Residual Network for Deepfake Video Detection](https://mlsanity.com/paper/2009.07480/) (2020)
- [Am I a Real or Fake Celebrity? Measuring Commercial Face Recognition Web APIs under Deepfake Impersonation Attack](https://mlsanity.com/paper/2103.00847/) (2021)

## Built on by (most cited first, 30 of 33)

- [FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Dataset](https://mlsanity.com/paper/2108.05080/) (2021)
- [CMUA-Watermark: A Cross-Model Universal Adversarial Watermark for Combating Deepfakes](https://mlsanity.com/paper/2105.10872/) (2021)
- [FReTAL: Generalizing Deepfake Detection using Knowledge Distillation and Representation Learning](https://mlsanity.com/paper/2105.13617/) (2021)
- [CoReD: Generalizing Fake Media Detection with Continual Representation using Distillation](https://mlsanity.com/paper/2107.02408/) (2021)
- [Evaluation of an Audio-Video Multimodal Deepfake Dataset using Unimodal and Multimodal Detectors](https://mlsanity.com/paper/2109.02993/) (2021)
- [TAR: Generalized Forensic Framework to Detect Deepfakes using Weakly Supervised Learning](https://mlsanity.com/paper/2105.06117/) (2021)
- [How well are open sourced AI-generated image detection models out-of-the-box: A comprehensive benchmark study](https://mlsanity.com/paper/2602.07814/) (2026)
- [SoK: Systematization and Benchmarking of Deepfake Detectors in a Unified Framework](https://mlsanity.com/paper/2401.04364/) (2024)
- [Why Do Facial Deepfake Detectors Fail?](https://mlsanity.com/paper/2302.13156/) (2023)
- [Deepfake in the Metaverse: Security Implications for Virtual Gaming, Meetings, and Offices](https://mlsanity.com/paper/2303.14612/) (2023)
- [The MeVer DeepFake Detection Service: Lessons Learnt from Developing and Deploying in the Wild](https://mlsanity.com/paper/2204.12816/) (2022)
- [Using Deep Learning to Detecting Deepfakes](https://mlsanity.com/paper/2207.13644/) (2022)
- [Towards an Awareness of Time Series Anomaly Detection Models' Adversarial Vulnerability](https://mlsanity.com/paper/2208.11264/) (2022)
- [UGAD: Universal Generative AI Detector utilizing Frequency Fingerprints](https://mlsanity.com/paper/2409.07913/) (2024)
- [Exploring the Impact of Moire Pattern on Deepfake Detectors](https://mlsanity.com/paper/2407.10399/) (2024)
- [LLMs Are Not Yet Ready for Deepfake Image Detection](https://mlsanity.com/paper/2506.10474/) (2025)
- [Feature Extraction Matters More: Universal Deepfake Disruption through Attacking Ensemble Feature Extractors](https://mlsanity.com/paper/2303.00200/) (2023)
- [How Generalizable are Deepfake Image Detectors? An Empirical Study](https://mlsanity.com/paper/2308.04177/) (2023)
- [MGFFD-VLM: Multi-Granularity Prompt Learning for Face Forgery Detection with VLM](https://mlsanity.com/paper/2507.12232/) (2025)
- [FakeParts: a New Family of AI-Generated DeepFakes](https://mlsanity.com/paper/2508.21052/) (2025)
- [FakeChain: Exposing Shallow Cues in Multi-Step Deepfake Detection](https://mlsanity.com/paper/2509.16602/) (2025)
- [Attention Consistency Refined Masked Frequency Forgery Representation for Generalizing Face Forgery Detection](https://mlsanity.com/paper/2307.11438/) (2023)
- [Visual Language Models as Zero-Shot Deepfake Detectors](https://mlsanity.com/paper/2507.22469/) (2025)
- [From Prediction to Explanation: Multimodal, Explainable, and Interactive Deepfake Detection Framework for Non-Expert Users](https://mlsanity.com/paper/2508.07596/) (2025)
- [Seeing Through the Blur: Unlocking Defocus Maps for Deepfake Detection](https://mlsanity.com/paper/2509.23289/) (2025)
- [Deepfake Detection for Facial Images with Facemasks](https://mlsanity.com/paper/2202.11359/) (2022)
- [Building an Invisible Shield for Your Portrait against Deepfakes](https://mlsanity.com/paper/2305.12881/) (2023)
- [LEAT: Towards Robust Deepfake Disruption in Real-World Scenarios via Latent Ensemble Attack](https://mlsanity.com/paper/2307.01520/) (2023)
- [Cross-Branch Orthogonality for Improved Generalization in Face Deepfake Detection](https://mlsanity.com/paper/2505.04888/) (2025)
- [Through the Lens: Benchmarking Deepfake Detectors Against Moir'e-Induced Distortions](https://mlsanity.com/paper/2510.23225/) (2025)

## Abstract (arXiv)

Deep learning-based video manipulation methods have become widely accessible to the masses. With little to no effort, people can quickly learn how to generate deepfake (DF) videos. While deep learning-based detection methods have been proposed to identify specific types of DFs, their performance suffers for other types of deepfake methods, including real-world deepfakes, on which they are not sufficiently trained. In other words, most of the proposed deep learning-based detection methods lack transferability and generalizability. Beyond detecting a single type of DF from benchmark deepfake datasets, we focus on developing a generalized approach to detect multiple types of DFs, including deepfakes from unknown generation methods such as DeepFake-in-the-Wild (DFW) videos. To better cope with unknown and unseen deepfakes, we introduce a Convolutional LSTM-based Residual Network (CLRNet), which adopts a unique model training strategy and explores spatial as well as the temporal information in deepfakes. Through extensive experiments, we show that existing defense methods are not ready for real-world deployment. Whereas our defense method (CLRNet) achieves far better generalization when detecting various benchmark deepfake methods (97.57% on average). Furthermore, we evaluate our approach with a high-quality DeepFake-in-the-Wild dataset, collected from the Internet containing numerous videos and having more than 150,000 frames. Our CLRNet model demonstrated that it generalizes well against high-quality DFW videos by achieving 93.86% detection accuracy, outperforming existing state-of-the-art defense methods by a considerable margin.