# DeepFake Detection Based on the Discrepancy Between the Face and its Context

arXiv: https://arxiv.org/abs/2008.12262 · HTML: https://mlsanity.com/paper/2008.12262/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 27 Aug 2020.

## Built on by (most cited first, 30 of 56)

- [Self-supervised Learning of Adversarial Example: Towards Good Generalizations for Deepfake Detection](https://mlsanity.com/paper/2203.12208/) (2022)
- [Implicit Identity Leakage: The Stumbling Block to Improving Deepfake Detection Generalization](https://mlsanity.com/paper/2210.14457/) (2022)
- [Semantic-Aware Implicit Neural Audio-Driven Video Portrait Generation](https://mlsanity.com/paper/2201.07786/) (2022)
- [FakeShield: Explainable Image Forgery Detection and Localization via Multi-modal Large Language Models](https://mlsanity.com/paper/2410.02761/) (2024)
- [TALL: Thumbnail Layout for Deepfake Video Detection](https://mlsanity.com/paper/2307.07494/) (2023)
- [Deepfake Generation and Detection: A Benchmark and Survey](https://mlsanity.com/paper/2403.17881/) (2024)
- [Deepfake Detection: A Comprehensive Survey from the Reliability Perspective](https://mlsanity.com/paper/2211.10881/) (2022)
- [Beyond the Prior Forgery Knowledge: Mining Critical Clues for General Face Forgery Detection](https://mlsanity.com/paper/2304.12489/) (2023)
- [Detect and Locate: Exposing Face Manipulation by Semantic- and Noise-level Telltales](https://mlsanity.com/paper/2107.05821/) (2021)
- [Proactive Image Manipulation Detection](https://mlsanity.com/paper/2203.15880/) (2022)
- [FSGANv2: Improved Subject Agnostic Face Swapping and Reenactment](https://mlsanity.com/paper/2202.12972/) (2022)
- [Fake It till You Make It: Curricular Dynamic Forgery Augmentations towards General Deepfake Detection](https://mlsanity.com/paper/2409.14444/) (2024)
- [Learning Spatiotemporal Inconsistency via Thumbnail Layout for Face Deepfake Detection](https://mlsanity.com/paper/2403.10261/) (2024)
- [LampMark: Proactive Deepfake Detection via Training-Free Landmark Perceptual Watermarks](https://mlsanity.com/paper/2411.17209/) (2024)
- [Digital and Physical Face Attacks: Reviewing and One Step Further](https://mlsanity.com/paper/2209.14692/) (2022)
- [Robust Identity Perceptual Watermark Against Deepfake Face Swapping](https://mlsanity.com/paper/2311.01357/) (2023)
- [Making DeepFakes more spurious: evading deep face forgery detection via trace removal attack](https://mlsanity.com/paper/2203.11433/) (2022)
- [The Age of Synthetic Realities: Challenges and Opportunities](https://mlsanity.com/paper/2306.11503/) (2023)
- [Exposing Lip-syncing Deepfakes from Mouth Inconsistencies](https://mlsanity.com/paper/2401.10113/) (2024)
- [Securing Deep Generative Models with Universal Adversarial Signature](https://mlsanity.com/paper/2305.16310/) (2023)
- [FedForgery: Generalized Face Forgery Detection with Residual Federated Learning](https://mlsanity.com/paper/2210.09563/) (2022)
- [Deepfake Media Generation and Detection in the Generative AI Era: A Survey and Outlook](https://mlsanity.com/paper/2411.19537/) (2024)
- [ReliableSwap: Boosting General Face Swapping Via Reliable Supervision](https://mlsanity.com/paper/2306.05356/) (2023)
- [A Multimodal Deviation Perceiving Framework for Weakly-Supervised Temporal Forgery Localization](https://mlsanity.com/paper/2507.16596/) (2025)
- [Detecting Lip-Syncing Deepfakes: Vision Temporal Transformer for Analyzing Mouth Inconsistencies](https://mlsanity.com/paper/2504.01470/) (2025)
- [Learning Real Facial Concepts for Independent Deepfake Detection](https://mlsanity.com/paper/2505.04460/) (2025)
- [Exploiting Facial Relationships and Feature Aggregation for Multi-Face Forgery Detection](https://mlsanity.com/paper/2310.04845/) (2023)
- [PIA: Deepfake Detection Using Phoneme-Temporal and Identity-Dynamic Analysis](https://mlsanity.com/paper/2510.14241/) (2025)
- [Exploring Spatial-Temporal Features for Deepfake Detection and Localization](https://mlsanity.com/paper/2210.15872/) (2022)
- [Weakly Supervised Training for Hologram Verification in Identity Documents](https://mlsanity.com/paper/2404.17253/) (2024)

## Abstract (arXiv)

We propose a method for detecting face swapping and other identity manipulations in single images. Face swapping methods, such as DeepFake, manipulate the face region, aiming to adjust the face to the appearance of its context, while leaving the context unchanged. We show that this modus operandi produces discrepancies between the two regions. These discrepancies offer exploitable telltale signs of manipulation. Our approach involves two networks: (i) a face identification network that considers the face region bounded by a tight semantic segmentation, and (ii) a context recognition network that considers the face context (e.g., hair, ears, neck). We describe a method which uses the recognition signals from our two networks to detect such discrepancies, providing a complementary detection signal that improves conventional real vs. fake classifiers commonly used for detecting fake images. Our method achieves state of the art results on the FaceForensics++, Celeb-DF-v2, and DFDC benchmarks for face manipulation detection, and even generalizes to detect fakes produced by unseen methods.