# D3: Scaling Up Deepfake Detection by Learning from Discrepancy

arXiv: https://arxiv.org/abs/2404.04584 · HTML: https://mlsanity.com/paper/2404.04584/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 48 references that are in the map.
- First posted on arXiv 6 Apr 2024.

## Builds on (most cited first, 30 of 40)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Zero-Shot Text-to-Image Generation](https://mlsanity.com/paper/2102.12092/) (2021)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Large Scale GAN Training for High Fidelity Natural Image Synthesis](https://mlsanity.com/paper/1809.11096/) (2018)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairs](https://mlsanity.com/paper/2111.02114/) (2021)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [FaceForensics++: Learning to Detect Manipulated Facial Images](https://mlsanity.com/paper/1901.08971/) (2019)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [CNN-generated images are surprisingly easy to spot... for now](https://mlsanity.com/paper/1912.11035/) (2019)
- [Vector Quantized Diffusion Model for Text-to-Image Synthesis](https://mlsanity.com/paper/2111.14822/) (2021)
- [DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation](https://mlsanity.com/paper/2110.02711/) (2021)
- [Towards Universal Fake Image Detectors that Generalize Across Generative Models](https://mlsanity.com/paper/2302.10174/) (2023)
- [Learning to See in the Dark](https://mlsanity.com/paper/1805.01934/) (2018)
- [DIRE for Diffusion-Generated Image Detection](https://mlsanity.com/paper/2303.09295/) (2023)
- [GenImage: A Million-Scale Benchmark for Detecting AI-Generated Image](https://mlsanity.com/paper/2306.08571/) (2023)
- [Detecting and Simulating Artifacts in GAN Fake Images](https://mlsanity.com/paper/1907.06515/) (2019)

## Built on by (most cited first, 30 of 45)

- [Toward Generalizable Forgery Detection and Reasoning](https://mlsanity.com/paper/2503.21210/) (2025)
- [Your One-Stop Solution for AI-Generated Video Detection](https://mlsanity.com/paper/2601.11035/) (2026)
- [Scaling Up AI-Generated Image Detection with Generator-Aware Prototypes](https://mlsanity.com/paper/2512.12982/) (2025)
- [NS-Net: Decoupling CLIP Semantic Information through NULL-Space for Generalizable AI-Generated Image Detection](https://mlsanity.com/paper/2508.01248/) (2025)
- [Simplicity Prevails: The Emergence of Generalizable AIGI Detection in Visual Foundation Models](https://mlsanity.com/paper/2602.01738/) (2026)
- [DGS-Net: Distillation-Guided Gradient Surgery for CLIP Fine-Tuning in AI-Generated Image Detection](https://mlsanity.com/paper/2511.13108/) (2025)
- [Fully AI-Generated Image Detection: Definition, Recent Advances and Challenges](https://mlsanity.com/paper/2502.19716/) (2025)
- [TriDF: Evaluating Perception, Detection, and Hallucination for Interpretable DeepFake Detection](https://mlsanity.com/paper/2512.10652/) (2025)
- [Layer Consistency Matters: Elegant Latent Transition Discrepancy for Generalizable Synthetic Image Detection](https://mlsanity.com/paper/2603.10598/) (2026)
- [FakeParts: a New Family of AI-Generated DeepFakes](https://mlsanity.com/paper/2508.21052/) (2025)
- [When Semantics Regulate: Rethinking Patch Shuffle and Internal Bias for Generated Image Detection with CLIP](https://mlsanity.com/paper/2511.19126/) (2025)
- [VIGIL: Part-Grounded Structured Reasoning for Generalizable Deepfake Detection](https://mlsanity.com/paper/2603.21526/) (2026)
- [LiteUpdate: A Lightweight Framework for Updating AI-Generated Image Detectors](https://mlsanity.com/paper/2511.07192/) (2025)
- [Robust and Calibrated Detection of Authentic Multimedia Content](https://mlsanity.com/paper/2512.15182/) (2025)
- [Quality-Aware Calibration for AI-Generated Image Detection in the Wild](https://mlsanity.com/paper/2604.15027/) (2026)
- [SSAFE: Simple and Strong AI-Generated Image Detection via Frozen Vision Encoders](https://mlsanity.com/paper/2606.08634/) (2026)
- [Veritas++: Value-aware On-Policy Distillation for Perception-Enhanced AIGI Detection](https://mlsanity.com/paper/2607.27113/) (2026)
- [Revealing the Implicit Noise-based Imprint of Generative Models](https://mlsanity.com/paper/2503.09314/) (2025)
- [Grab-3D: Detecting AI-Generated Videos from 3D Geometric Temporal Consistency](https://mlsanity.com/paper/2512.13665/) (2025)
- [HEDGE: Heterogeneous Ensemble for Detection of AI-GEnerated Images in the Wild](https://mlsanity.com/paper/2604.03555/) (2026)
- [LOGER: Local--Global Ensemble for Robust Deepfake Detection in the Wild](https://mlsanity.com/paper/2604.03558/) (2026)
- [HydraPrompt: An Adaptive and Asymmetric Framework of Vision-Language Models for Synthetic Image Detection](https://mlsanity.com/paper/2605.26421/) (2026)
- [DetectZoo: A Unified Toolkit for AI-Generated Content Detection Across Text, Audio, and Image Modalities](https://mlsanity.com/paper/2606.04205/) (2026)
- [LaP-Forensics: Latent-Pixel Consistency Guided Multimodal Reasoning for Deepfake Detection](https://mlsanity.com/paper/2607.25962/) (2026)
- [Proto-LeakNet: Towards Signal-Leak Aware Attribution in Synthetic Human Face Imagery](https://mlsanity.com/paper/2511.04260/) (2025)
- [How Noise Benefits AI-generated Image Detection](https://mlsanity.com/paper/2511.16136/) (2025)
- [Beyond Flicker: Detecting Kinematic Inconsistencies for Generalizable Deepfake Video Detection](https://mlsanity.com/paper/2512.04175/) (2025)
- [Open Set Face Forgery Detection via Dual-Level Evidence Collection](https://mlsanity.com/paper/2512.04331/) (2025)
- [Na"ive Exposure of Generative AI Capabilities Undermines Deepfake Detection](https://mlsanity.com/paper/2603.10504/) (2026)
- [Generalizable Detection of AI Generated Images with Large Models and Fuzzy Decision Tree](https://mlsanity.com/paper/2603.28508/) (2026)

## Abstract (arXiv)

The boom of Generative AI brings opportunities entangled with risks and concerns. Existing literature emphasizes the generalization capability of deepfake detection on unseen generators, significantly promoting the detector's ability to identify more universal artifacts. This work seeks a step toward a universal deepfake detection system with better generalization and robustness. We do so by first scaling up the existing detection task setup from the one-generator to multiple-generators in training, during which we disclose two challenges presented in prior methodological designs and demonstrate the divergence of detectors' performance. Specifically, we reveal that the current methods tailored for training on one specific generator either struggle to learn comprehensive artifacts from multiple generators or sacrifice their fitting ability for seen generators (i.e., In-Domain (ID) performance) to exchange the generalization for unseen generators (i.e., Out-Of-Domain (OOD) performance). To tackle the above challenges, we propose our Discrepancy Deepfake Detector (D3) framework, whose core idea is to deconstruct the universal artifacts from multiple generators by introducing a parallel network branch that takes a distorted image feature as an extra discrepancy signal and supplement its original counterpart. Extensive scaled-up experiments demonstrate the effectiveness of D3, achieving 5.3% accuracy improvement in the OOD testing compared to the current SOTA methods while maintaining the ID performance. The source code will be updated in our GitHub repository: https://github.com/BigAandSmallq/D3