# Locate and Verify: A Two-Stream Network for Improved Deepfake Detection

arXiv: https://arxiv.org/abs/2309.11131 · HTML: https://mlsanity.com/paper/2309.11131/

## Key facts

- Cited by 26 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 61 references that are in the map.
- First posted on arXiv 20 Sep 2023.

## Builds on (most cited first, 30 of 46)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [Learning deep representations by mutual information estimation and maximization](https://mlsanity.com/paper/1808.06670/) (2018)
- [FaceForensics++: Learning to Detect Manipulated Facial Images](https://mlsanity.com/paper/1901.08971/) (2019)
- [A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild](https://mlsanity.com/paper/2008.10010/) (2020)
- [Face2Face: Real-time Face Capture and Reenactment of RGB Videos](https://mlsanity.com/paper/2007.14808/) (2020)
- [Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensics](https://mlsanity.com/paper/1909.12962/) (2019)
- [Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Clues](https://mlsanity.com/paper/2007.09355/) (2020)
- [Leveraging Frequency Analysis for Deep Fake Image Recognition](https://mlsanity.com/paper/2003.08685/) (2020)
- [Face X-ray for More General Face Forgery Detection](https://mlsanity.com/paper/1912.13458/) (2019)
- [Multi-attentional Deepfake Detection](https://mlsanity.com/paper/2103.02406/) (2021)
- [Detecting Deepfakes with Self-Blended Images](https://mlsanity.com/paper/2204.08376/) (2022)
- [FSGAN: Subject Agnostic Face Swapping and Reenactment](https://mlsanity.com/paper/1908.05932/) (2019)
- [Exposing DeepFake Videos By Detecting Face Warping Artifacts](https://mlsanity.com/paper/1811.00656/) (2018)
- [Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detection](https://mlsanity.com/paper/2012.07657/) (2020)
- [Generalizing Face Forgery Detection with High-frequency Features](https://mlsanity.com/paper/2103.12376/) (2021)
- [On the Detection of Digital Face Manipulation](https://mlsanity.com/paper/1910.01717/) (2019)
- [DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detection](https://mlsanity.com/paper/2001.03024/) (2020)
- [Deferred Neural Rendering: Image Synthesis using Neural Textures](https://mlsanity.com/paper/1904.12356/) (2019)
- [Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domain](https://mlsanity.com/paper/2103.01856/) (2021)
- [SimSwap: An Efficient Framework For High Fidelity Face Swapping](https://mlsanity.com/paper/2106.06340/) (2021)
- [DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detection](https://mlsanity.com/paper/2001.00179/) (2020)
- [The Deepfake Detection Challenge (DFDC) Preview Dataset](https://mlsanity.com/paper/1910.08854/) (2019)
- [Learning Self-Consistency for Deepfake Detection](https://mlsanity.com/paper/2012.09311/) (2020)
- [F3Net: Fusion, Feedback and Focus for Salient Object Detection](https://mlsanity.com/paper/1911.11445/) (2019)
- [Self-supervised Learning of Adversarial Example: Towards Good Generalizations for Deepfake Detection](https://mlsanity.com/paper/2203.12208/) (2022)
- [Two-branch Recurrent Network for Isolating Deepfakes in Videos](https://mlsanity.com/paper/2008.03412/) (2020)
- [Exploring Temporal Coherence for More General Video Face Forgery Detection](https://mlsanity.com/paper/2108.06693/) (2021)
- [Frequency-aware Discriminative Feature Learning Supervised by Single-Center Loss for Face Forgery Detection](https://mlsanity.com/paper/2103.09096/) (2021)

## Built on by (most cited first, 26 of 26)

- [Exposing the Deception: Uncovering More Forgery Clues for Deepfake Detection](https://mlsanity.com/paper/2403.01786/) (2024)
- [Lips Are Lying: Spotting the Temporal Inconsistency between Audio and Visual in Lip-Syncing DeepFakes](https://mlsanity.com/paper/2401.15668/) (2024)
- [Delocate: Detection and Localization for Deepfake Videos with Randomly-Located Tampered Traces](https://mlsanity.com/paper/2401.13516/) (2024)
- [DiMoDif: Discourse Modality-information Differentiation for Audio-visual Deepfake Detection and Localization](https://mlsanity.com/paper/2411.10193/) (2024)
- [Joint-Motion Mutual Learning for Pose Estimation in Videos](https://mlsanity.com/paper/2408.02285/) (2024)
- [DeepShield: Fortifying Deepfake Video Detection with Local and Global Forgery Analysis](https://mlsanity.com/paper/2510.25237/) (2025)
- [Evolving from Single-modal to Multi-modal Facial Deepfake Detection: Progress and Challenges](https://mlsanity.com/paper/2406.06965/) (2024)
- [Exploiting Facial Relationships and Feature Aggregation for Multi-Face Forgery Detection](https://mlsanity.com/paper/2310.04845/) (2023)
- [Passive Deepfake Detection Across Multi-modalities: A Comprehensive Survey](https://mlsanity.com/paper/2411.17911/) (2024)
- [Harnessing Frequency Spectrum Insights for Image Copyright Protection Against Diffusion Models](https://mlsanity.com/paper/2503.11071/) (2025)
- [AuViRe: Audio-visual Speech Representation Reconstruction for Deepfake Temporal Localization](https://mlsanity.com/paper/2511.18993/) (2025)
- [SpatioTemporal Learning for Human Pose Estimation in Sparsely-Labeled Videos](https://mlsanity.com/paper/2501.15073/) (2025)
- [HVIS: A Human-like Vision and Inference System for Human Motion Prediction](https://mlsanity.com/paper/2502.16913/) (2025)
- [SEED: A Benchmark Dataset for Sequential Facial Attribute Editing with Diffusion Models](https://mlsanity.com/paper/2506.00562/) (2025)
- [Exploring the Collaborative Advantage of Low-level Information on Generalizable AI-Generated Image Detection](https://mlsanity.com/paper/2504.00463/) (2025)
- [When Detectors Forget Forensics: Blocking Semantic Shortcuts for Generalizable AI-Generated Image Detection](https://mlsanity.com/paper/2603.09242/) (2026)
- [ETGuard: Malicious Encrypted Traffic Detection in Blockchain-based Power Grid Systems](https://mlsanity.com/paper/2408.10657/) (2024)
- [Multi-Grained Feature Pruning for Video-Based Human Pose Estimation](https://mlsanity.com/paper/2503.05365/) (2025)
- [Contrastive Desensitization Learning for Cross Domain Face Forgery Detection](https://mlsanity.com/paper/2505.20675/) (2025)
- [Unmasking Synthetic Realities in Generative AI: A Comprehensive Review of Adversarially Robust Deepfake Detection Systems](https://mlsanity.com/paper/2507.21157/) (2025)
- [Morphology-optimized Multi-Scale Fusion: Combining Local Artifacts and Mesoscopic Semantics for Deepfake Detection and Localization](https://mlsanity.com/paper/2509.13776/) (2025)
- [Shallow- and Deep-fake Image Manipulation Localization Using Vision Mamba and Guided Graph Neural Network](https://mlsanity.com/paper/2601.02566/) (2026)
- [R2BD: A Reconstruction-Based Method for Generalizable and Efficient Detection of Fake Images](https://mlsanity.com/paper/2601.08867/) (2026)
- [Unveiling Deepfakes: A Frequency-Aware Triple Branch Network for Deepfake Detection](https://mlsanity.com/paper/2604.17477/) (2026)
- [Mind the Rift: Cross-Scale Coupling Mismatch for AI-Generated Video Detection](https://mlsanity.com/paper/2609.00742/) (2026)
- [Residual Transferability in Neural Image Watermarking](https://mlsanity.com/paper/2609.32241/) (2026)

## Abstract (arXiv)

Deepfake has taken the world by storm, triggering a trust crisis. Current deepfake detection methods are typically inadequate in generalizability, with a tendency to overfit to image contents such as the background, which are frequently occurring but relatively unimportant in the training dataset. Furthermore, current methods heavily rely on a few dominant forgery regions and may ignore other equally important regions, leading to inadequate uncovering of forgery cues. In this paper, we strive to address these shortcomings from three aspects: (1) We propose an innovative two-stream network that effectively enlarges the potential regions from which the model extracts forgery evidence. (2) We devise three functional modules to handle the multi-stream and multi-scale features in a collaborative learning scheme. (3) Confronted with the challenge of obtaining forgery annotations, we propose a Semi-supervised Patch Similarity Learning strategy to estimate patch-level forged location annotations. Empirically, our method demonstrates significantly improved robustness and generalizability, outperforming previous methods on six benchmarks, and improving the frame-level AUC on Deepfake Detection Challenge preview dataset from 0.797 to 0.835 and video-level AUC on CelebDF_v1 dataset from 0.811 to 0.847. Our implementation is available at https://github.com/sccsok/Locate-and-Verify.