Quantitative Metrics for Evaluating Explanations of Video DeepFake Detectors
Not on the map. Why?
Abstract (arXiv)
The proliferation of DeepFake technology is a rising challenge in today's society, owing to more powerful and accessible generation methods. To counter this, the research community has developed detectors of ever-increasing accuracy. However, the ability to explain the decisions of such models to users is lacking behind and is considered an accessory in large-scale benchmarks, despite being a crucial requirement for the correct deployment of automated tools for content moderation. We attribute the issue to the reliance on qualitative comparisons and the lack of established metrics. We describe a simple set of metrics to evaluate the visual quality and informativeness of explanations of video DeepFake classifiers from a human-centric perspective. With these metrics, we compare common approaches to improve explanation quality and discuss their effect on both classification and explanation performance on the recent DFDC and DFD datasets.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 63 of its 99 references that are in the map. First posted on arXiv 7 Oct 2022.
Builds on
63 of 99 references.
References · 63
- 2017Attention Is All You Needcited by 59,932
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2016Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localizationcited by 6,690
- 2017A Unified Approach to Interpreting Model Predictionscited by 6,488
- 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
- 2016"Why Should I Trust You?": Explaining the Predictions of Any Classifiercited by 5,040
- 2017Axiomatic Attribution for Deep Networkscited by 3,350
- 2015Learning Deep Features for Discriminative Localizationcited by 2,430
- 2013Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Mapscited by 2,348
- 2013Visualizing and Understanding Convolutional Networkscited by 2,278
- 2017The Kinetics Human Action Video Datasetcited by 1,894
- 2017Improved Regularization of Convolutional Neural Networks with Cutoutcited by 1,623
38 more
- 2021ViViT: A Video Vision Transformercited by 1,610
- 2019Explainable Artificial Intelligence (XAI): Concepts, Taxonomies, Opportunities and Challenges toward Responsible AIcited by 1,543
- 2018ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustnesscited by 1,430
- 2015Show, Attend and Tell: Neural Image Caption Generation with Visual Attentioncited by 1,421
- 2017Explanation in Artificial Intelligence: Insights from the Social Sciencescited by 1,201
- 2017SmoothGrad: removing noise by adding noisecited by 1,097
- 2019FaceForensics++: Learning to Detect Manipulated Facial Imagescited by 1,095
- 2014Striving for Simplicity: The All Convolutional Netcited by 983
- 2019Attention is not Explanationcited by 916
- 2021Multiscale Vision Transformerscited by 896
- 2016Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networkscited by 895
- 2018RISE: Randomized Input Sampling for Explanation of Black-box Modelscited by 825
- 2020Quantifying Attention Flow in Transformerscited by 759
- 2020Face2Face: Real-time Face Capture and Reenactment of RGB Videoscited by 631
- 2017Interpretable Explanations of Black Boxes by Meaningful Perturbationcited by 611
- 2019Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensicscited by 573
- 2017Rethinking Spatiotemporal Feature Learning: Speed-Accuracy Trade-offs in Video Classificationcited by 521
- 2018Explaining Explanations: An Overview of Interpretability of Machine Learningcited by 508
- 2018Towards Robust Interpretability with Self-Explaining Neural Networkscited by 505
- 2015Evaluating the visualization of what a Deep Neural Network has learnedcited by 416
- 2018MesoNet: a Compact Facial Video Forgery Detection Networkcited by 409
- 2020HateXplain: A Benchmark Dataset for Explainable Hate Speech Detectioncited by 403
- 2017Methods for Interpreting and Understanding Deep Neural Networkscited by 342
- 2019FSGAN: Subject Agnostic Face Swapping and Reenactmentcited by 319
- 2018Exposing DeepFake Videos By Detecting Face Warping Artifactscited by 318
- 2021Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformerscited by 316
- 2018Exposing Deep Fakes Using Inconsistent Head Posescited by 294
- 2020BiSeNet V2: Bilateral Network with Guided Aggregation for Real-time Semantic Segmentationcited by 294
- 2019On the Detection of Digital Face Manipulationcited by 259
- 2015Explaining NonLinear Classification Decisions with Deep Taylor Decompositioncited by 248
- 2020DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detectioncited by 231
- 2018Talking Face Generation by Adversarially Disentangled Audio-Visual Representationcited by 212
- 2021WildDeepfake: A Challenging Real-World Dataset for Deepfake Detectioncited by 207
- 2016Seed, Expand and Constrain: Three Principles for Weakly-Supervised Image Segmentationcited by 198
- 2018Capsule-Forensics: Using Capsule Networks to Detect Forged Images and Videoscited by 180
- 2019Multi-task Learning For Detecting and Segmenting Manipulated Facial Images and Videoscited by 130
- 2019XRAI: Better Attributions Through Regionscited by 116
- 2021Are Convolutional Neural Networks or Transformers more like human vision?cited by 109
50 most cited of 63 All 63 (JSON)
Built on by
Citing papers per year: 2022: 1; 2023: 1; 2024: 1.
3 papers.