# ProtoExplorer: Interpretable Forensic Analysis of Deepfake Videos using Prototype Exploration and Refinement

arXiv: https://arxiv.org/abs/2309.11155 · HTML: https://mlsanity.com/paper/2309.11155/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 75 references that are in the map.
- First posted on arXiv 20 Sep 2023.

## Builds on (most cited first, 30 of 38)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- ["Why Should I Trust You?": Explaining the Predictions of Any Classifier](https://mlsanity.com/paper/1602.04938/) (2016)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://mlsanity.com/paper/1802.03426/) (2018)
- [Learning Important Features Through Propagating Activation Differences](https://mlsanity.com/paper/1704.02685/) (2017)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [Explanation in Artificial Intelligence: Insights from the Social Sciences](https://mlsanity.com/paper/1706.07269/) (2017)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [FaceForensics++: Learning to Detect Manipulated Facial Images](https://mlsanity.com/paper/1901.08971/) (2019)
- [Joint Face Detection and Alignment using Multi-task Cascaded Convolutional Networks](https://mlsanity.com/paper/1604.02878/) (2016)
- [This Looks Like That: Deep Learning for Interpretable Image Recognition](https://mlsanity.com/paper/1806.10574/) (2018)
- [Face2Face: Real-time Face Capture and Reenactment of RGB Videos](https://mlsanity.com/paper/2007.14808/) (2020)
- [Interpretable Machine Learning: Fundamental Principles and 10 Grand Challenges](https://mlsanity.com/paper/2103.11251/) (2021)
- [Deferred Neural Rendering: Image Synthesis using Neural Textures](https://mlsanity.com/paper/1904.12356/) (2019)
- [Deep Learning for Case-Based Reasoning through Prototypes: A Neural Network that Explains Its Predictions](https://mlsanity.com/paper/1710.04806/) (2017)
- [Neural Prototype Trees for Interpretable Fine-grained Image Recognition](https://mlsanity.com/paper/2012.02046/) (2020)
- [Media Forensics and DeepFakes: an overview](https://mlsanity.com/paper/2001.06564/) (2020)
- [Layer-wise Relevance Propagation for Neural Networks with Local Renormalization Layers](https://mlsanity.com/paper/1604.00825/) (2016)
- [DeepFaceLab: Integrated, flexible and extensible face-swapping framework](https://mlsanity.com/paper/2005.05535/) (2020)
- [Explainable Deep Learning: A Field Guide for the Uninitiated](https://mlsanity.com/paper/2004.14545/) (2020)
- [Deformable ProtoPNet: An Interpretable Image Classifier Using Deformable Prototypes](https://mlsanity.com/paper/2111.15000/) (2021)
- [XProtoNet: Diagnosis in Chest Radiography with Global and Local Explanations](https://mlsanity.com/paper/2103.10663/) (2021)
- [Summit: Scaling Deep Learning Interpretability by Visualizing Activation and Attribution Summarizations](https://mlsanity.com/paper/1904.02323/) (2019)
- [Interpretable and Steerable Sequence Learning via Prototypes](https://mlsanity.com/paper/1907.09728/) (2019)
- [explAIner: A Visual Analytics Framework for Interactive and Explainable Machine Learning](https://mlsanity.com/paper/1908.00087/) (2019)
- [Seq2Seq-Vis: A Visual Debugging Tool for Sequence-to-Sequence Models](https://mlsanity.com/paper/1804.09299/) (2018)
- [Interpretable Image Recognition with Hierarchical Prototypes](https://mlsanity.com/paper/1906.10651/) (2019)
- [CNN Explainer: Learning Convolutional Neural Networks with Interactive Visualization](https://mlsanity.com/paper/2004.15004/) (2020)
- [ActiVis: Visual Exploration of Industry-Scale Deep Neural Network Models](https://mlsanity.com/paper/1704.01942/) (2017)
- [A Survey of Visual Analytics Techniques for Machine Learning](https://mlsanity.com/paper/2008.09632/) (2020)

## Built on by (most cited first, 5 of 5)

- [This Looks Better than That: Better Interpretable Models with ProtoPNeXt](https://mlsanity.com/paper/2406.14675/) (2024)
- [PUDD: Towards Robust Multi-modal Prototype-based Deepfake Detection](https://mlsanity.com/paper/2406.15921/) (2024)
- [XAI-CF -- Examining the Role of Explainable Artificial Intelligence in Cyber Forensics](https://mlsanity.com/paper/2402.02452/) (2024)
- [Why Fake ? Unveiling the Semantic Vocabulary of Deepfake Detectors](https://mlsanity.com/paper/2607.07216/) (2026)
- [DF-CBM: Region-Aware Concept Bottleneck Models for Deepfake Detection](https://mlsanity.com/paper/2609.35096/) (2026)

## Abstract (arXiv)

In high-stakes settings, Machine Learning models that can provide predictions that are interpretable for humans are crucial. This is even more true with the advent of complex deep learning based models with a huge number of tunable parameters. Recently, prototype-based methods have emerged as a promising approach to make deep learning interpretable. We particularly focus on the analysis of deepfake videos in a forensics context. Although prototype-based methods have been introduced for the detection of deepfake videos, their use in real-world scenarios still presents major challenges, in that prototypes tend to be overly similar and interpretability varies between prototypes. This paper proposes a Visual Analytics process model for prototype learning, and, based on this, presents ProtoExplorer, a Visual Analytics system for the exploration and refinement of prototype-based deepfake detection models. ProtoExplorer offers tools for visualizing and temporally filtering prototype-based predictions when working with video data. It disentangles the complexity of working with spatio-temporal prototypes, facilitating their visualization. It further enables the refinement of models by interactively deleting and replacing prototypes with the aim to achieve more interpretable and less biased predictions while preserving detection accuracy. The system was designed with forensic experts and evaluated in a number of rounds based on both open-ended think aloud evaluation and interviews. These sessions have confirmed the strength of our prototype based exploration of deepfake videos while they provided the feedback needed to continuously improve the system.