mlsanityData to 7 Oct 2026

Evolving from Single-modal to Multi-modal Facial Deepfake Detection: Progress and Challenges

arXiv 2406.06965 ↗First posted 11 Jun 2024cs.CVcited by 6

On the map: Multimodal LLMs / vision-language models → MLLM jailbreaks and adversarial robustness

Abstract (arXiv)

As synthetic media, including video, audio, and text, become increasingly indistinguishable from real content, the risks of misinformation, identity fraud, and social manipulation escalate. This survey traces the evolution of deepfake detection from early single-modal methods to sophisticated multi-modal approaches that integrate audio-visual and text-visual cues. We present a structured taxonomy of detection techniques and analyze the transition from GAN-based to diffusion model-driven deepfakes, which introduce new challenges due to their heightened realism and robustness against detection. Unlike prior surveys that primarily focus on single-modal detection or earlier deepfake techniques, this work provides the most comprehensive study to date, encompassing the latest advancements in multi-modal deepfake detection, generalization challenges, proactive defense mechanisms, and emerging datasets specifically designed to support new interpretability and reasoning tasks. We further explore the role of Vision-Language Models (VLMs) and Multimodal Large Language Models (MLLMs) in strengthening detection robustness against increasingly sophisticated deepfake attacks. By systematically categorizing existing methods and identifying emerging research directions, this survey serves as a foundation for future advancements in combating AI-generated facial forgeries. A curated list of all related papers can be found at hrefhttps://github.com/qiqitao77/Comprehensive-Advances-in-Deepfake-Detection-Spanning-Diverse-Modalitieshttps://github.com/qiqitao77/Awesome-Comprehensive-Deepfake-Detection.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Placed in MLLM jailbreaks and adversarial robustness, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 144 of its 265 references that are in the map. First posted on arXiv 11 Jun 2024.

Builds on

144 of 265 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Can ChatGPT Detect DeepFakes? A Study of Using Multimodal Large Language Models for Media Forensics21 Mar
  2. 2024SHIELD : An Evaluation Benchmark for Face Spoofing and Forgery Detection with Multimodal Large Language Models6 Feb

Other references · 142

  1. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  4. 2023Visual Instruction Tuningcited by 7,792
  5. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  6. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  7. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  8. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  9. 2023InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuningcited by 2,855
  10. 2020Knowledge Distillation: A Surveycited by 1,578
  11. 2019FaceForensics++: Learning to Detect Manipulated Facial Imagescited by 1,095
  12. 2022Diffusion Models in Vision: A Surveycited by 1,008
38 more
  1. 2019An Introduction to Variational Autoencoderscited by 997
  2. 2023Vision-Language Models for Vision Tasks: A Surveycited by 834
  3. 2023CogVLM: Visual Expert for Pretrained Language Modelscited by 704
  4. 2019Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensicscited by 573
  5. 2020Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Cluescited by 518
  6. 2019Face X-ray for More General Face Forgery Detectioncited by 469
  7. 2018MesoNet: a Compact Facial Video Forgery Detection Networkcited by 409
  8. 2023DIRE for Diffusion-Generated Image Detectioncited by 386
  9. 2023Chat-UniVi: Unified Visual Representation Empowers Large Language Models with Image and Video Understandingcited by 368
  10. 2021Multi-attentional Deepfake Detectioncited by 339
  11. 2022Detecting Deepfakes with Self-Blended Imagescited by 332
  12. 2019FSGAN: Subject Agnostic Face Swapping and Reenactmentcited by 319
  13. 2023The Stable Signature: Rooting Watermarks in Latent Diffusion Modelscited by 289
  14. 2020Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detectioncited by 286
  15. 2022On the detection of synthetic images generated by diffusion modelscited by 282
  16. 2021Generalizing Face Forgery Detection with High-frequency Featurescited by 261
  17. 2019On the Detection of Digital Face Manipulationcited by 259
  18. 2020DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detectioncited by 251
  19. 2021Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domaincited by 241
  20. 2020Global Texture Enhancement for Fake Face Detection in the Wildcited by 220
  21. 2021WildDeepfake: A Challenging Real-World Dataset for Deepfake Detectioncited by 207
  22. 2023A Survey on Self-supervised Learning: Algorithms, Applications, and Future Trendscited by 205
  23. 2020Learning Self-Consistency for Deepfake Detectioncited by 179
  24. 2021FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Datasetcited by 179
  25. 2023UCF: Uncovering Common Features for Generalizable Deepfake Detectioncited by 177
  26. 2022Self-supervised Learning of Adversarial Example: Towards Good Generalizations for Deepfake Detectioncited by 172
  27. 2021Exploring Temporal Coherence for More General Video Face Forgery Detectioncited by 158
  28. 2020Domain Adaptive Ensemble Learningcited by 156
  29. 2023A Recipe for Watermarking Diffusion Modelscited by 136
  30. 2023Hierarchical Fine-Grained Image Forgery Detection and Localizationcited by 134
  31. 2021Frequency-aware Discriminative Feature Learning Supervised by Single-Center Loss for Face Forgery Detectioncited by 133
  32. 2022Leveraging Real Talking Faces via Self-Supervision for Robust Forgery Detectioncited by 130
  33. 2023Transcending Forgery Specificity with Latent Space Augmentation for Generalizable Deepfake Detectioncited by 126
  34. 2021ForgeryNet: A Versatile Benchmark for Comprehensive Forgery Analysiscited by 124
  35. 2022Implicit Identity Leakage: The Stumbling Block to Improving Deepfake Detection Generalizationcited by 122
  36. 2023Intriguing properties of synthetic images: from generative adversarial networks to diffusion modelscited by 121
  37. 2022Towards the Detection of Diffusion Model Deepfakescited by 111
  38. 2023EditGuard: Versatile Image Watermarking for Tamper Localization and Copyright Protectioncited by 109

50 most cited of 142 All 142 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 3; 2026: 3.

6 papers.

Similar, not cited

Close in wording, no citation either way

  1. 2025PIA: Deepfake Detection Using Phoneme-Temporal and Identity-Dynamic Analysis
  2. 2025Generating and Detecting Various Types of Fake Image and Audio Content: A Review of Modern Deep Learning Technologies and Tools
  3. 2025LLMs Are Not Yet Ready for Deepfake Image Detection
  4. 2024The Tug-of-War Between Deepfake Generation and Detection
  5. 2025Rethinking Vision-Language Model in Face Forensics: Multi-Modal Interpretable Forged Face Detector
  6. 2023Parents and Children: Distinguishing Multimodal DeepFakes from Natural Images
  7. 2025Visual Language Models as Zero-Shot Deepfake Detectors
  8. 2025Multi-modal Deepfake Detection and Localization with FPN-Transformer