mlsanityData to 14 Oct 2026

Common Sense Reasoning for Deepfake Detection

arXiv 2402.00126 ↗First posted 31 Jan 2024cs.CV cs.CLcited by 65

Not on the map. Why?

Abstract (arXiv)

State-of-the-art deepfake detection approaches rely on image-based features extracted via neural networks. While these approaches trained in a supervised manner extract likely fake features, they may fall short in representing unnatural `non-physical' semantic facial attributes -- blurry hairlines, double eyebrows, rigid eye pupils, or unnatural skin shading. However, such facial attributes are easily perceived by humans and used to discern the authenticity of an image based on human common sense. Furthermore, image-based feature extraction methods that provide visual explanations via saliency maps can be hard to interpret for humans. To address these challenges, we frame deepfake detection as a Deepfake Detection VQA (DD-VQA) task and model human intuition by providing textual explanations that describe common sense reasons for labeling an image as real or fake. We introduce a new annotated dataset and propose a Vision and Language Transformer-based framework for the DD-VQA task. We also incorporate text and image-aware feature alignment formulation to enhance multi-modal representation learning. As a result, we improve upon existing deepfake detection models by integrating our learned vision representations, which reason over common sense knowledge from the DD-VQA task. We provide extensive empirical results demonstrating that our method enhances detection performance, generalization ability, and language-based interpretability in the deepfake detection task.

Key facts. Cited by 65 papers in the map (data through 14 Oct 2026). Builds on 49 of its 67 references that are in the map. First posted on arXiv 31 Jan 2024.

Builds on

49 of 67 references.

References · 49

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  5. 2018Representation Learning with Contrastive Predictive Codingcited by 7,777
  6. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,733
  7. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  8. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  9. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  10. 2022Generative Adversarial Networkscited by 5,627
  11. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
  12. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
37 more
  1. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  2. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  3. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  4. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  5. 2015VQA: Visual Question Answeringcited by 2,506
  6. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,371
  7. 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,332
  8. 2014CIDEr: Consensus-based Image Description Evaluationcited by 2,046
  9. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,464
  10. 2019FaceForensics++: Learning to Detect Manipulated Facial Imagescited by 1,101
  11. 2017Vision-and-Language Navigation: Interpreting visually-grounded navigation instructions in real environmentscited by 1,027
  12. 2016SPICE: Semantic Propositional Image Caption Evaluationcited by 824
  13. 2022A-OKVQA: A Benchmark for Visual Question Answering using World Knowledgecited by 710
  14. 2019Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensicscited by 575
  15. 2018From Recognition to Cognition: Visual Commonsense Reasoningcited by 556
  16. 2022Visual Programming: Compositional visual reasoning without trainingcited by 516
  17. 2019Face X-ray for More General Face Forgery Detectioncited by 470
  18. 2018nocaps: novel object captioning at scalecited by 444
  19. 2024InstantID: Zero-shot Identity-Preserving Generation in Secondscited by 436
  20. 2022Detecting Deepfakes with Self-Blended Imagescited by 338
  21. 2020The Creation and Detection of Deepfakes: A Surveycited by 247
  22. 2020DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detectioncited by 233
  23. 2018A Simple Method for Commonsense Reasoningcited by 223
  24. 2021WildDeepfake: A Challenging Real-World Dataset for Deepfake Detectioncited by 209
  25. 2023Hierarchical Fine-Grained Image Forgery Detection and Localizationcited by 134
  26. 2022Leveraging Real Talking Faces via Self-Supervision for Robust Forgery Detectioncited by 130
  27. 2016Show and Tell: Lessons learned from the 2015 MSCOCO Image Captioning Challengecited by 116
  28. 2022Towards the Detection of Diffusion Model Deepfakescited by 111
  29. 2021Combining EfficientNet and Vision Transformers for Video Deepfake Detectioncited by 82
  30. 2020Use HiResCAM instead of Grad-CAM for faithful explanations of convolutional neural networkscited by 79
  31. 2023Towards General Visual-Linguistic Face Forgery Detectioncited by 45
  32. 2022Multi-domain Learning for Updating Face Anti-spoofing Modelscited by 39
  33. 2022Detecting and Recovering Sequential DeepFake Manipulationcited by 34
  34. 2023VLN-Trans: Translator for the Vision and Language Navigation Agentcited by 25
  35. 2023Controllable Guide-Space for Generalizable Face Forgery Detectioncited by 21
  36. 2023Tracing Hyperparameter Dependencies for Model Parsing via Learnable Graph Pooling Networkcited by 8
  37. 2022LOViS: Learning Orientation and Visual Signals for Vision and Language Navigationcited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 14; 2025: 26; 2026: 25.

65 papers.

Sort:

By area:

  1. 2024FakeShield: Explainable Image Forgery Detection and Localization via Multi-modal Large Language Modelscited by 113
  2. 2024SIDA: Social Media Image Deepfake Detection, Localization and Explanation with Large Multimodal Modelcited by 108
  3. 2025Spot the Fake: Large Multimodal Model-Based Synthetic Image Detection with Artifact Explanationcited by 68
  4. 2024FFAA: Multimodal Large Language Model based Explainable Open-World Face Forgery Analysis Assistantcited by 61
  5. 2024On Learning Multi-Modal Forgery Representation for Diffusion Generated Video Detectioncited by 40
  6. 2025Rethinking Vision-Language Model in Face Forensics: Multi-Modal Interpretable Forged Face Detectorcited by 40
  7. 2024FakeBench: Probing Explainable Fake Image Detection via Large Multimodal Modelscited by 36
  8. 2024X2-DFD: A framework for eXplainable and eXtendable Deepfake Detectioncited by 35
  9. 2025AIGI-Holmes: Towards Explainable and Generalizable AI-Generated Image Detection via Multimodal Large Language Modelscited by 33
  10. 2024SPARTUN3D: Situated Spatial Understanding of 3D World in Large Language Modelscited by 27
  11. 2025Toward Generalizable Forgery Detection and Reasoningcited by 20
  12. 2024Language-guided Hierarchical Fine-grained Image Forgery Detection and Localizationcited by 16
38 more
  1. 202550 Years of Automated Face Recognitioncited by 15
  2. 2026When and How Much to Imagine: Adaptive Test-Time Scaling with World Models for Visual Spatial Reasoningcited by 14
  3. 2024ForgeryGPT: A Multimodal LLM for Interpretable Image Forgery Detection and Localizationcited by 13
  4. 2025Forensics-Bench: A Comprehensive Forgery Detection Benchmark Suite for Large Vision Language Modelscited by 13
  5. 2025Face-Human-Bench: A Comprehensive Benchmark of Face and Human Understanding for Multi-modal Assistantscited by 12
  6. 2025VLForgery Face Triad: Detection, Localization and Attribution via Multimodal Large Language Modelscited by 12
  7. 2024MFCLIP: Multi-modal Fine-grained CLIP for Generalizable Diffusion Face Forgery Detectioncited by 11
  8. 2025FakeScope: Large Multimodal Expert Model for Transparent AI-Generated Image Forensicscited by 11
  9. 2025A Quality-Guided Mixture of Score-Fusion Experts Framework for Human Recognitioncited by 11
  10. 2024A Review of Multimodal Explainable Artificial Intelligence: Past, Present and Futurecited by 9
  11. 2025Towards Explainable Fake Image Detection with Multi-Modal Large Language Modelscited by 8
  12. 2026Fake-HR1: Rethinking Reasoning of Vision Language Model for Synthetic Image Detectioncited by 8
  13. 2025Guard Me If You Know Me: Protecting Specific Face-Identity from Deepfakescited by 7
  14. 2024Evolving from Single-modal to Multi-modal Facial Deepfake Detection: Progress and Challengescited by 6
  15. 2026EvoGuard: An Extensible Agentic RL-based Framework for Practical and Evolving AI-Generated Image Detectioncited by 6
  16. 2025So-Fake: Benchmarking and Explaining Social Media Image Forgery Detectioncited by 5
  17. 2025PRPO: Paragraph-level Policy Optimization for Vision-Language Deepfake Detectioncited by 5
  18. 2026On the Holistic Approach for Detecting Human Image Forgerycited by 5
  19. 2024A Hitchhikers Guide to Fine-Grained Face Forgery Detection Using Common Sense Reasoningcited by 4
  20. 2025TriDF: Evaluating Perception, Detection, and Hallucination for Interpretable DeepFake Detectioncited by 4
  21. 2025Code-in-the-Loop Forensics: Agentic Tool Use for Image Forgery Detectioncited by 4
  22. 2026MARE: Multimodal Alignment and Reinforcement for Explainable Deepfake Detection via Vision-Language Modelscited by 4
  23. 2026FakeVLM-R1: Internalizing Physical Laws via CoT for Synthetic Image Detectioncited by 4
  24. 2025MGFFD-VLM: Multi-Granularity Prompt Learning for Face Forgery Detection with VLMcited by 3
  25. 2025FakeParts: a New Family of AI-Generated DeepFakescited by 3
  26. 2026Explainable Deepfake Detection with RL Enhanced Self-Blended Imagescited by 3
  27. 2026VIGIL: Part-Grounded Structured Reasoning for Generalizable Deepfake Detectioncited by 3
  28. 2025A Data-Driven Diffusion-based Approach for Audio Deepfake Explanationscited by 2
  29. 2025EDVD-LLaMA: Explainable Deepfake Video Detection via Multimodal Large Language Model Reasoningcited by 2
  30. 2025Fake-in-Facext: Towards Fine-Grained Explainable DeepFake Analysiscited by 2
  31. 2026VRAG-DFD: Verifiable Retrieval-Augmentation for MLLM-based Deepfake Detectioncited by 2
  32. 2025AuthGuard: Generalizable Deepfake Detection via Language Guidancecited by 1
  33. 2025CorrDetail: Visual Detail Enhanced Self-Correction for Face Forgery Detectioncited by 1
  34. 2026Deepfake detectors are DUMB: A benchmark to assess adversarial training robustness under transferability constraintscited by 1
  35. 2026EvolveReason: Self-Evolving Reasoning Paradigm for Explainable Deepfake Facial Image Identificationcited by 1
  36. 2026Perception, Verdict, and Evolution: Hindsight-Driven Self-Refining Forensics Agent for AI-Generated Image Detectioncited by 1
  37. 2026XPlainVerse: A Million-Scale Benchmark for Explainable Deepfake Detectioncited by 1
  38. 2024Revisiting Face Forgery Detection: From Facial Representation to Forgery Detectioncited by 0

50 most cited of 65 All 65 (JSON)