# Towards More General Video-based Deepfake Detection through Facial Component Guided Adaptation for Foundation Model

arXiv: https://arxiv.org/abs/2404.05583 · HTML: https://mlsanity.com/paper/2404.05583/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 57 references that are in the map.
- First posted on arXiv 8 Apr 2024.

## Builds on (most cited first, 30 of 45)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Visual Prompt Tuning](https://mlsanity.com/paper/2203.12119/) (2022)
- [Understanding intermediate layers using linear classifier probes](https://mlsanity.com/paper/1610.01644/) (2016)
- [Alias-Free Generative Adversarial Networks](https://mlsanity.com/paper/2106.12423/) (2021)
- [Reproducible scaling laws for contrastive language-image learning](https://mlsanity.com/paper/2212.07143/) (2022)
- [FaceForensics++: Learning to Detect Manipulated Facial Images](https://mlsanity.com/paper/1901.08971/) (2019)
- [CNN-generated images are surprisingly easy to spot... for now](https://mlsanity.com/paper/1912.11035/) (2019)
- [Face2Face: Real-time Face Capture and Reenactment of RGB Videos](https://mlsanity.com/paper/2007.14808/) (2020)
- [Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensics](https://mlsanity.com/paper/1909.12962/) (2019)
- [Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Clues](https://mlsanity.com/paper/2007.09355/) (2020)
- [Scaling up GANs for Text-to-Image Synthesis](https://mlsanity.com/paper/2303.05511/) (2023)
- [Towards Universal Fake Image Detectors that Generalize Across Generative Models](https://mlsanity.com/paper/2302.10174/) (2023)
- [Face X-ray for More General Face Forgery Detection](https://mlsanity.com/paper/1912.13458/) (2019)
- [Emergent Correspondence from Image Diffusion](https://mlsanity.com/paper/2306.03881/) (2023)
- [DIRE for Diffusion-Generated Image Detection](https://mlsanity.com/paper/2303.09295/) (2023)
- [Detecting Deepfakes with Self-Blended Images](https://mlsanity.com/paper/2204.08376/) (2022)
- [Side Adapter Network for Open-Vocabulary Semantic Segmentation](https://mlsanity.com/paper/2302.12242/) (2023)
- [Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detection](https://mlsanity.com/paper/2012.07657/) (2020)
- [DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detection](https://mlsanity.com/paper/2001.03024/) (2020)
- [Deferred Neural Rendering: Image Synthesis using Neural Textures](https://mlsanity.com/paper/1904.12356/) (2019)
- [Lip Reading Sentences in the Wild](https://mlsanity.com/paper/1611.05358/) (2016)

## Built on by (most cited first, 30 of 38)

- [Deepfake Detection that Generalizes Across Benchmarks](https://mlsanity.com/paper/2508.06248/) (2025)
- [VideoVeritas: AI-Generated Video Detection via Perception Pretext Reinforcement Learning](https://mlsanity.com/paper/2602.08828/) (2026)
- [Deepfake Media Generation and Detection in the Generative AI Era: A Survey and Outlook](https://mlsanity.com/paper/2411.19537/) (2024)
- [Semantics-Oriented Multitask Learning for DeepFake Detection: A Joint Embedding Approach](https://mlsanity.com/paper/2408.16305/) (2024)
- [AuViRe: Audio-visual Speech Representation Reconstruction for Deepfake Temporal Localization](https://mlsanity.com/paper/2511.18993/) (2025)
- [TriDF: Evaluating Perception, Detection, and Hallucination for Interpretable DeepFake Detection](https://mlsanity.com/paper/2512.10652/) (2025)
- [Forensics Adapter: Unleashing CLIP for Generalizable Face Forgery Detection](https://mlsanity.com/paper/2411.19715/) (2024)
- [Fair and Interpretable Deepfake Detection in Videos](https://mlsanity.com/paper/2510.17264/) (2025)
- [AVFakeBench: A Comprehensive Audio-Video Forgery Detection Benchmark for AV-LMMs](https://mlsanity.com/paper/2511.21251/) (2025)
- [ExposeAnyone: Personalized Audio-to-Expression Diffusion Models Are Robust Zero-Shot Face Forgery Detectors](https://mlsanity.com/paper/2601.02359/) (2026)
- [Deepfake detectors are DUMB: A benchmark to assess adversarial training robustness under transferability constraints](https://mlsanity.com/paper/2601.05986/) (2026)
- [When Detectors Forget Forensics: Blocking Semantic Shortcuts for Generalizable AI-Generated Image Detection](https://mlsanity.com/paper/2603.09242/) (2026)
- [LOGER: Local--Global Ensemble for Robust Deepfake Detection in the Wild](https://mlsanity.com/paper/2604.03558/) (2026)
- [Face-D(^2)CL: Multi-Domain Synergistic Representation with Dual Continual Learning for Facial DeepFake Detection](https://mlsanity.com/paper/2604.08159/) (2026)
- [Uncertainty-Aware Deepfake Detection via Multi-View Structural Learning](https://mlsanity.com/paper/2607.28769/) (2026)
- [PATE-Forensics: Perception-as-Tool for Explainable Deepfake Forensics with General-Purpose MLLMs](https://mlsanity.com/paper/2608.18573/) (2026)
- [Revisiting Face Forgery Detection: From Facial Representation to Forgery Detection](https://mlsanity.com/paper/2409.16945/) (2024)
- [DiffusionFF: A Diffusion-based Framework for Joint Face Forgery Detection and Fine-Grained Artifact Localization](https://mlsanity.com/paper/2508.01873/) (2025)
- [LAKAN: Landmark-assisted Adaptive Kolmogorov-Arnold Network for Face Forgery Detection](https://mlsanity.com/paper/2510.00634/) (2025)
- [Cognitive Inception: Agentic Reasoning against Visual Deceptions by Injecting Skepticism](https://mlsanity.com/paper/2511.17672/) (2025)
- [OmniFD: A Unified Model for Versatile Face Forgery Detection](https://mlsanity.com/paper/2512.01128/) (2025)
- [Beyond Flicker: Detecting Kinematic Inconsistencies for Generalizable Deepfake Video Detection](https://mlsanity.com/paper/2512.04175/) (2025)
- [Low-rank Orthogonal Subspace Intervention for Generalizable Face Forgery Detection](https://mlsanity.com/paper/2601.11915/) (2026)
- [Beyond Static Artifacts: A Forensic Benchmark for Video Deepfake Reasoning in Vision Language Models](https://mlsanity.com/paper/2602.21779/) (2026)
- [Towards Generalizable Deepfake Detection via Real Distribution Bias Correction](https://mlsanity.com/paper/2603.14005/) (2026)
- [Unleashing Vision-Language Semantics for Deepfake Video Detection](https://mlsanity.com/paper/2603.24454/) (2026)
- [Robust Deepfake Detection: Mitigating Spatial Attention Drift via Calibrated Complementary Ensembles](https://mlsanity.com/paper/2604.25889/) (2026)
- [Detecting Deepfakes via Hamiltonian Dynamics](https://mlsanity.com/paper/2605.04405/) (2026)
- [Exposing and Mitigating Temporal Attack in Deepfake Video Detection](https://mlsanity.com/paper/2605.07398/) (2026)
- [Deepfake Detection in Social Media: A Temporal Artifact Analysis Using 3D Convolutional Neural Networks](https://mlsanity.com/paper/2605.17573/) (2026)

## Abstract (arXiv)

Generative models have enabled the creation of highly realistic facial-synthetic images, raising significant concerns due to their potential for misuse. Despite rapid advancements in the field of deepfake detection, developing efficient approaches to leverage foundation models for improved generalizability to unseen forgery samples remains challenging. To address this challenge, we propose a novel side-network-based decoder that extracts spatial and temporal cues using the CLIP image encoder for generalized video-based Deepfake detection. Additionally, we introduce Facial Component Guidance (FCG) to enhance spatial learning generalizability by encouraging the model to focus on key facial regions. By leveraging the generic features of a vision-language foundation model, our approach demonstrates promising generalizability on challenging Deepfake datasets while also exhibiting superiority in training data efficiency, parameter efficiency, and model robustness.