mlsanityData to 7 Oct 2026

Explaining Deepfake Detection by Analysing Image Matching

arXiv 2207.09679 ↗First posted 20 Jul 2022cs.CVcited by 34

Not on the map. Why?

Abstract (arXiv)

This paper aims to interpret how deepfake detection models learn artifact features of images when just supervised by binary labels. To this end, three hypotheses from the perspective of image matching are proposed as follows. 1. Deepfake detection models indicate real/fake images based on visual concepts that are neither source-relevant nor target-relevant, that is, considering such visual concepts as artifact-relevant. 2. Besides the supervision of binary labels, deepfake detection models implicitly learn artifact-relevant visual concepts through the FST-Matching (i.e. the matching fake, source, target images) in the training set. 3. Implicitly learned artifact visual concepts through the FST-Matching in the raw training set are vulnerable to video compression. In experiments, the above hypotheses are verified among various DNNs. Furthermore, based on this understanding, we propose the FST-Matching Deepfake Detection Model to boost the performance of forgery detection on compressed videos. Experiment results show that our method achieves great performance, especially on highly-compressed (e.g. c40) videos.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). Builds on 41 of its 60 references that are in the map. First posted on arXiv 20 Jul 2022.

Builds on

41 of 60 references.

References · 41

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
  3. 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
  4. 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
  5. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,415
  6. 2017Squeeze-and-Excitation Networkscited by 4,763
  7. 2015Learning Deep Features for Discriminative Localizationcited by 2,430
  8. 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
  9. 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,320
  10. 2013Visualizing and Understanding Convolutional Networkscited by 2,278
  11. 2019FaceForensics++: Learning to Detect Manipulated Facial Imagescited by 1,095
  12. 2017Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networkscited by 936
29 more
  1. 2017Network Dissection: Quantifying Interpretability of Deep Visual Representationscited by 703
  2. 2020Face2Face: Real-time Face Capture and Reenactment of RGB Videoscited by 631
  3. 2017Interpretable Explanations of Black Boxes by Meaningful Perturbationcited by 611
  4. 2019Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensicscited by 573
  5. 2019Face X-ray for More General Face Forgery Detectioncited by 469
  6. 2018MesoNet: a Compact Facial Video Forgery Detection Networkcited by 409
  7. 2014Understanding Deep Image Representations by Inverting Themcited by 399
  8. 2021Multi-attentional Deepfake Detectioncited by 339
  9. 2015Understanding Neural Networks Through Deep Visualizationcited by 332
  10. 2020Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detectioncited by 286
  11. 2020What makes fake images detectable? Understanding properties that generalizecited by 272
  12. 2021Generalizing Face Forgery Detection with High-frequency Featurescited by 261
  13. 2021Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domaincited by 241
  14. 2019FaceShifter: Towards High Fidelity And Occlusion Aware Face Swappingcited by 221
  15. 2020Learning Self-Consistency for Deepfake Detectioncited by 179
  16. 2014Object Detectors Emerge in Deep Scene CNNscited by 178
  17. 2015Inverting Visual Representations with Convolutional Networkscited by 168
  18. 2021Frequency-aware Discriminative Feature Learning Supervised by Single-Center Loss for Face Forgery Detectioncited by 133
  19. 2018Net2Vec: Quantifying and Explaining how Concepts are Encoded by Filters in Deep Neural Networkscited by 128
  20. 2020The DeepFake Detection Challenge (DFDC) Datasetcited by 125
  21. 2019Explaining Deep Neural Networks with a Polynomial Time Algorithm for Shapley Values Approximationcited by 115
  22. 2018Interpreting CNNs via Decision Treescited by 89
  23. 2020Explaining Knowledge Distillation by Quantifying the Knowledgecited by 65
  24. 2019Use of a Capsule Network to Detect Fake Images and Videoscited by 63
  25. 2021Exploiting Fine-grained Face Forgery Clues via Progressive Enhancement Learningcited by 60
  26. 2021Improving the Efficiency and Robustness of Deepfakes Detection through Precise Geometric Featurescited by 57
  27. 2017Recasting Residual-based Local Descriptors as Convolutional Neural Networks: an Application to Image Forgery Detectioncited by 56
  28. 2017Interpreting CNN Knowledge via an Explanatory Graphcited by 49
  29. 2020Interpreting and Boosting Dropout from a Game-Theoretic Viewcited by 34

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 3; 2023: 7; 2024: 11; 2025: 10; 2026: 3.

34 papers.

Sort:

By area:

  1. 2022Implicit Identity Leakage: The Stumbling Block to Improving Deepfake Detection Generalizationcited by 122
  2. 2024Can ChatGPT Detect DeepFakes? A Study of Using Multimodal Large Language Models for Media Forensicscited by 68
  3. 2022SeeABLE: Soft Discrepancies and Bounded Contrastive Learning for Exposing Deepfakescited by 49
  4. 2022Deepfake Detection: A Comprehensive Survey from the Reliability Perspectivecited by 48
  5. 2024Exposing the Deception: Uncovering More Forgery Clues for Deepfake Detectioncited by 47
  6. 2024FakeBench: Probing Explainable Fake Image Detection via Large Multimodal Modelscited by 36
  7. 2025Ivy-Fake: A Unified Explainable Framework and Benchmark for Image and Video AIGC Detectioncited by 34
  8. 2024DiffusionFake: Enhancing Generalization in Deepfake Detection via Guided Stable Diffusioncited by 29
  9. 2023Contrastive Pseudo Learning for Open-World DeepFake Attributioncited by 27
  10. 2023Locate and Verify: A Two-Stream Network for Improved Deepfake Detectioncited by 26
  11. 2025From Specificity to Generality: Revisiting Generalizable Artifacts in Detecting Face Deepfakescited by 14
  12. 2025FreqDebias: Towards Generalizable Deepfake Detection via Consistency-Driven Frequency Debiasingcited by 14
22 more
  1. 2025Forensics-Bench: A Comprehensive Forgery Detection Benchmark Suite for Large Vision Language Modelscited by 13
  2. 2024Delocate: Detection and Localization for Deepfake Videos with Randomly-Located Tampered Tracescited by 12
  3. 2024A Survey of Defenses Against AI-Generated Visual Media: Detection,Disruption, and Authenticationcited by 12
  4. 2024Deepfake Media Generation and Detection in the Generative AI Era: A Survey and Outlookcited by 10
  5. 2025INTER: Mitigating Hallucination in Large Vision-Language Models by Interaction Guidance Samplingcited by 10
  6. 2023An adversarial attack approach for eXplainable AI evaluation on deepfake detection modelscited by 7
  7. 2024Diagnosing the Compositional Knowledge of Vision Language Models from a Game-Theoretic Viewcited by 7
  8. 2023CrossDF: Improving Cross-Domain Deepfake Detection with Deep Information Decompositioncited by 6
  9. 2024Passive Deepfake Detection Across Multi-modalities: A Comprehensive Surveycited by 4
  10. 2024A Large-scale Universal Evaluation Benchmark For Face Forgery Detectioncited by 3
  11. 2023Towards generalizing deep-audio fake detection networkscited by 2
  12. 2023Building an Invisible Shield for Your Portrait against Deepfakescited by 1
  13. 2025Cross-Branch Orthogonality for Improved Generalization in Face Deepfake Detectioncited by 1
  14. 2025CorrDetail: Visual Detail Enhanced Self-Correction for Face Forgery Detectioncited by 1
  15. 2025Fair-FLIP: Fair Deepfake Detection with Fairness-Oriented Final Layer Input Prioritisingcited by 1
  16. 2023Recap: Detecting Deepfake Video with Unpredictable Tampered Traces via Recovering Faces and Mapping Recovered Facescited by 0
  17. 2024Revisiting Face Forgery Detection: From Facial Representation to Forgery Detectioncited by 0
  18. 2025Towards Generalizable Deepfake Detection with Spatial-Frequency Collaborative Learning and Hierarchical Cross-Modal Fusioncited by 0
  19. 2025Fair and Interpretable Deepfake Detection in Videoscited by 0
  20. 2026EvolveReason: Self-Evolving Reasoning Paradigm for Explainable Deepfake Facial Image Identificationcited by 0
  21. 2026SynMulti: Synthetic-to-Real Learning for Multimodal Video Understandingcited by 0
  22. 2026DF-MoE: Generalizable Deepfake Detection via Multimodal Sparse Mixture-of-Expertscited by 0