Segmentation-Guided Spatial Indexing for Generalizable and Explainable Deepfake Detection
Not on the map. Why?
Abstract (arXiv)
We introduce segmentation-guided spatial indexing for generalizable and explainable deepfake detection. The key idea reverses the standard design order: rather than pooling all facial tokens and classifying afterward, we first select semantically meaningful patch tokens, then pool only those. A frozen FaRL parser assigns each DINOv3 ViT-L/16 patch token a semantic label; non-target tokens are discarded; a linear probe classifies the retained region. This spatial indexing exploits DINOv3's patch-level spatial consistency, the same property that enables emergent segmentation, to present the probe with a purer regional subspace where manipulation-relevant evidence is less diluted by whole-face cues. Region attribution is structural: when the mouth model predicts fake, the decision used only mouth tokens, not an overlaid saliency map. On Celeb-DF v2, the mouth-indexed probe achieves AUC 0.905, outperforming LipForensics (+8.1 pp) and Xception (+16.9 pp), with no DINOv3 or FaRL fine-tuning and no target-domain data. Ablations isolate the mechanism: replacing regional selection with DINOv3's CLS token drops Celeb-DF v2 AUC by 26.4 pp; replacing DINOv3 with FaRL features drops it by 20.9 pp. Both DINOv3 representation and the spatial index are independently necessary; neither alone approaches the full system.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 9 of its 13 references that are in the map. First posted on arXiv 25 May 2026.
Builds on
9 of 13 references.
References · 9
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
- 2019FaceForensics++: Learning to Detect Manipulated Facial Imagescited by 1,101
- 2019Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensicscited by 575
- 2023Towards Universal Fake Image Detectors that Generalize Across Generative Modelscited by 490
- 2021Multi-attentional Deepfake Detectioncited by 341
- 2020Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detectioncited by 287
- 2021General Facial Representation Learning in a Visual-Linguistic Mannercited by 195
- 2025Unlocking the Hidden Potential of CLIP in Generalizable Deepfake Detectioncited by 10
- 2026Face2Parts: Exploring Coarse-to-Fine Inter-Regional Facial Dependencies for Generalized Deepfake Detectioncited by 4
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Towards More General Video-based Deepfake Detection through Facial Component Guided Adaptation for Foundation Model
- 2026Look Before You Judge: Training-Free Region Mining for Grounded and Explainable Deepfake Detection
- 2026DF-MoE: Generalizable Deepfake Detection via Multimodal Sparse Mixture-of-Experts
- 2024Evolving from Single-modal to Multi-modal Facial Deepfake Detection: Progress and Challenges
- 2024Deepfake Generation and Detection: A Benchmark and Survey
- 2026Beyond Static Artifacts: A Forensic Benchmark for Video Deepfake Reasoning in Vision Language Models
- 2026VIGIL: Part-Grounded Structured Reasoning for Generalizable Deepfake Detection
- 2026Cross-Domain Generalization Limits of Vision Foundation Models in Facial Deepfake Detection