Evolving from Single-modal to Multi-modal Facial Deepfake Detection: Progress and Challenges
On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness
Abstract (arXiv)
As synthetic media, including video, audio, and text, become increasingly indistinguishable from real content, the risks of misinformation, identity fraud, and social manipulation escalate. This survey traces the evolution of deepfake detection from early single-modal methods to sophisticated multi-modal approaches that integrate audio-visual and text-visual cues. We present a structured taxonomy of detection techniques and analyze the transition from GAN-based to diffusion model-driven deepfakes, which introduce new challenges due to their heightened realism and robustness against detection. Unlike prior surveys that primarily focus on single-modal detection or earlier deepfake techniques, this work provides the most comprehensive study to date, encompassing the latest advancements in multi-modal deepfake detection, generalization challenges, proactive defense mechanisms, and emerging datasets specifically designed to support new interpretability and reasoning tasks. We further explore the role of Vision-Language Models (VLMs) and Multimodal Large Language Models (MLLMs) in strengthening detection robustness against increasingly sophisticated deepfake attacks. By systematically categorizing existing methods and identifying emerging research directions, this survey serves as a foundation for future advancements in combating AI-generated facial forgeries. A curated list of all related papers can be found at hrefhttps://github.com/qiqitao77/Comprehensive-Advances-in-Deepfake-Detection-Spanning-Diverse-Modalitieshttps://github.com/qiqitao77/Awesome-Comprehensive-Deepfake-Detection.
Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 144 of its 265 references that are in the map. First posted on arXiv 11 Jun 2024.
Builds on
144 of 265 references.
Closest ancestors · 2
Same direction, newest first
- 2024Can ChatGPT Detect DeepFakes? A Study of Using Multimodal Large Language Models for Media Forensics21 Mar
- 2024SHIELD : An Evaluation Benchmark for Face Spoofing and Forgery Detection with Multimodal Large Language Models6 Feb
Other references · 142
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
- 2023Visual Instruction Tuningcited by 7,792
- 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
- 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
- 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
- 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
- 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
- 2020Knowledge Distillation: A Surveycited by 1,578
- 2019FaceForensics++: Learning to Detect Manipulated Facial Imagescited by 1,095
- 2022Diffusion Models in Vision: A Surveycited by 1,008
38 more
- 2019An Introduction to Variational Autoencoderscited by 997
- 2023Vision-Language Models for Vision Tasks: A Surveycited by 834
- 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 704
- 2019Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensicscited by 573
- 2020Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Cluescited by 518
- 2019Face X-ray for More General Face Forgery Detectioncited by 469
- 2018MesoNet: a Compact Facial Video Forgery Detection Networkcited by 409
- 2023DIRE for Diffusion-Generated Image Detectioncited by 386
- 2023Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understandingcited by 368
- 2021Multi-attentional Deepfake Detectioncited by 339
- 2022Detecting Deepfakes with Self-Blended Imagescited by 332
- 2019FSGAN: Subject Agnostic Face Swapping and Reenactmentcited by 319
- 2023The Stable Signature: Rooting Watermarks in Latent Diffusion Modelscited by 289
- 2020Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detectioncited by 286
- 2022On the detection of synthetic images generated by diffusion modelscited by 282
- 2021Generalizing Face Forgery Detection with High-frequency Featurescited by 261
- 2019On the Detection of Digital Face Manipulationcited by 259
- 2020DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detectioncited by 251
- 2021Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domaincited by 241
- 2020Global Texture Enhancement for Fake Face Detection in the Wildcited by 220
- 2021WildDeepfake: A Challenging Real-World Dataset for Deepfake Detectioncited by 207
- 2023A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trendscited by 205
- 2020Learning Self-Consistency for Deepfake Detectioncited by 179
- 2021FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Datasetcited by 179
- 2023UCF: Uncovering Common Features for Generalizable Deepfake Detectioncited by 177
- 2022Self-supervised Learning of Adversarial Example: Towards Good Generalizations for Deepfake Detectioncited by 172
- 2021Exploring Temporal Coherence for More General Video Face Forgery Detectioncited by 158
- 2020Domain Adaptive Ensemble Learningcited by 156
- 2023A Recipe for Watermarking Diffusion Modelscited by 136
- 2023Hierarchical Fine-Grained Image Forgery Detection and Localizationcited by 134
- 2021Frequency-aware Discriminative Feature Learning Supervised by Single-Center Loss for Face Forgery Detectioncited by 133
- 2022Leveraging Real Talking Faces via Self-Supervision for Robust Forgery Detectioncited by 130
- 2023Transcending Forgery Specificity with Latent Space Augmentation for Generalizable Deepfake Detectioncited by 126
- 2021ForgeryNet: A Versatile Benchmark for Comprehensive Forgery Analysiscited by 124
- 2022Implicit Identity Leakage: The Stumbling Block to Improving Deepfake Detection Generalizationcited by 122
- 2023Intriguing properties of synthetic images: from generative adversarial networks to diffusion modelscited by 121
- 2022Towards the Detection of Diffusion Model Deepfakescited by 111
- 2023EditGuard: Versatile Image Watermarking for Tamper Localization and Copyright Protectioncited by 109
50 most cited of 142 All 142 (JSON)
Built on by
Citing papers per year: 2025: 3; 2026: 3.
6 papers.
- 2025Can Multi-modal (reasoning) LLMs work as deepfake detectors?cited by 14
- 2025TalkingHeadBench: A Multi-Modal Benchmark & Analysis of Talking-Head DeepFake Detectioncited by 10
- 2025ForenX: Towards Explainable AI-Generated Image Detection with Multimodal Large Language Modelscited by 4
- 2026SpecSem-Net: Integrating Spectral and Semantic Features for Robust AI-generated Video Detectioncited by 1
- 2026Are DeepFakes Realistic Enough? Exploring Semantic Mismatch as a Novel Challengecited by 0
- 2026Deepfake Detection in Social Media: A Temporal Artifact Analysis Using 3D Convolutional Neural Networkscited by 0
- 2026Deepfake Detection in Social Media: A Temporal Artifact Analysis Using 3D Convolutional Neural Networks17 May 2026
- 2026SpecSem-Net: Integrating Spectral and Semantic Features for Robust AI-generated Video Detection17 May 2026
- 2026Are DeepFakes Realistic Enough? Exploring Semantic Mismatch as a Novel Challenge30 Apr 2026
- 2025ForenX: Towards Explainable AI-Generated Image Detection with Multimodal Large Language Models2 Aug 2025
- 2025TalkingHeadBench: A Multi-Modal Benchmark & Analysis of Talking-Head DeepFake Detection30 May 2025
- 2025Can Multi-modal (reasoning) LLMs work as deepfake detectors?25 Mar 2025
Similar, not cited
Close in wording, no citation either way
- 2025PIA: Deepfake Detection Using Phoneme-Temporal and Identity-Dynamic Analysis
- 2025Generating and Detecting Various Types of Fake Image and Audio Content: A Review of Modern Deep Learning Technologies and Tools
- 2025LLMs Are Not Yet Ready for Deepfake Image Detection
- 2024The Tug-of-War Between Deepfake Generation and Detection
- 2025Rethinking Vision-Language Model in Face Forensics: Multi-Modal Interpretable Forged Face Detector
- 2023Parents and Children: Distinguishing Multimodal DeepFakes from Natural Images
- 2025Visual Language Models as Zero-Shot Deepfake Detectors
- 2025Multi-modal Deepfake Detection and Localization with FPN-Transformer