# Local Relation Learning for Face Forgery Detection

arXiv: https://arxiv.org/abs/2105.02577 · HTML: https://mlsanity.com/paper/2105.02577/

## Key facts

- Cited by 100 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 29 references that are in the map.
- First posted on arXiv 6 May 2021.

## Builds on (most cited first, 21 of 21)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [FaceForensics++: Learning to Detect Manipulated Facial Images](https://mlsanity.com/paper/1901.08971/) (2019)
- [Face2Face: Real-time Face Capture and Reenactment of RGB Videos](https://mlsanity.com/paper/2007.14808/) (2020)
- [Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensics](https://mlsanity.com/paper/1909.12962/) (2019)
- [Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Clues](https://mlsanity.com/paper/2007.09355/) (2020)
- [Face X-ray for More General Face Forgery Detection](https://mlsanity.com/paper/1912.13458/) (2019)
- [MesoNet: a Compact Facial Video Forgery Detection Network](https://mlsanity.com/paper/1809.00888/) (2018)
- [Detecting and Simulating Artifacts in GAN Fake Images](https://mlsanity.com/paper/1907.06515/) (2019)
- [HRank: Filter Pruning using High-Rank Feature Map](https://mlsanity.com/paper/2002.10179/) (2020)
- [On the Detection of Digital Face Manipulation](https://mlsanity.com/paper/1910.01717/) (2019)
- [Global Texture Enhancement for Fake Face Detection in the Wild](https://mlsanity.com/paper/2002.00133/) (2020)
- [The Deepfake Detection Challenge (DFDC) Preview Dataset](https://mlsanity.com/paper/1910.08854/) (2019)
- [Capsule-Forensics: Using Capsule Networks to Detect Forged Images and Videos](https://mlsanity.com/paper/1810.11215/) (2018)
- [Two-branch Recurrent Network for Isolating Deepfakes in Videos](https://mlsanity.com/paper/2008.03412/) (2020)
- [Multi-task Learning For Detecting and Segmenting Manipulated Facial Images and Videos](https://mlsanity.com/paper/1906.06876/) (2019)
- [Unmasking DeepFakes with simple Features](https://mlsanity.com/paper/1911.00686/) (2019)
- [Image Splicing Localization Using A Multi-Task Fully Convolutional Network (MFCN)](https://mlsanity.com/paper/1709.02016/) (2017)
- [Recasting Residual-based Local Descriptors as Convolutional Neural Networks: an Application to Image Forgery Detection](https://mlsanity.com/paper/1703.04615/) (2017)

## Built on by (most cited first, 30 of 100)

- [Detecting Deepfakes with Self-Blended Images](https://mlsanity.com/paper/2204.08376/) (2022)
- [Forgery-aware Adaptive Transformer for Generalizable Synthetic Image Detection](https://mlsanity.com/paper/2312.16649/) (2023)
- [UCF: Uncovering Common Features for Generalizable Deepfake Detection](https://mlsanity.com/paper/2304.13949/) (2023)
- [Implicit Identity Leakage: The Stumbling Block to Improving Deepfake Detection Generalization](https://mlsanity.com/paper/2210.14457/) (2022)
- [Dual Contrastive Learning for General Face Forgery Detection](https://mlsanity.com/paper/2112.13522/) (2021)
- [Detecting and Grounding Multi-Modal Media Manipulation](https://mlsanity.com/paper/2304.02556/) (2023)
- [LAA-Net: Localized Artifact Attention Network for Quality-Agnostic and Generalizable Deepfake Detection](https://mlsanity.com/paper/2401.13856/) (2024)
- [Spatiotemporal Inconsistency Learning for DeepFake Video Detection](https://mlsanity.com/paper/2109.01860/) (2021)
- [ObjectFormer for Image Manipulation Detection and Localization](https://mlsanity.com/paper/2203.14681/) (2022)
- [CORE: Consistent Representation Learning for Face Forgery Detection](https://mlsanity.com/paper/2206.02749/) (2022)
- [UIA-ViT: Unsupervised Inconsistency-Aware Method based on Vision Transformer for Face Forgery Detection](https://mlsanity.com/paper/2210.12752/) (2022)
- [Exploiting Fine-grained Face Forgery Clues via Progressive Enhancement Learning](https://mlsanity.com/paper/2112.13977/) (2021)
- [Exploiting Style Latent Flows for Generalizing Deepfake Video Detection](https://mlsanity.com/paper/2403.06592/) (2024)
- [CMUA-Watermark: A Cross-Model Universal Adversarial Watermark for Combating Deepfakes](https://mlsanity.com/paper/2105.10872/) (2021)
- [Detecting and Grounding Multi-Modal Media Manipulation and Beyond](https://mlsanity.com/paper/2309.14203/) (2023)
- [Deepfake Detection: A Comprehensive Survey from the Reliability Perspective](https://mlsanity.com/paper/2211.10881/) (2022)
- [Beyond the Prior Forgery Knowledge: Mining Critical Clues for General Face Forgery Detection](https://mlsanity.com/paper/2304.12489/) (2023)
- [FreqBlender: Enhancing DeepFake Detection by Blending Frequency Knowledge](https://mlsanity.com/paper/2404.13872/) (2024)
- [Exploring Frequency Adversarial Attacks for Face Forgery Detection](https://mlsanity.com/paper/2203.15674/) (2022)
- [Towards General Visual-Linguistic Face Forgery Detection](https://mlsanity.com/paper/2307.16545/) (2023)
- [Delving into Sequential Patches for Deepfake Detection](https://mlsanity.com/paper/2207.02803/) (2022)
- [SHIELD : An Evaluation Benchmark for Face Spoofing and Forgery Detection with Multimodal Large Language Models](https://mlsanity.com/paper/2402.04178/) (2024)
- [Rethinking Vision-Language Model in Face Forensics: Multi-Modal Interpretable Forged Face Detector](https://mlsanity.com/paper/2503.20188/) (2025)
- [Exploring Disentangled Content Information for Face Forgery Detection](https://mlsanity.com/paper/2207.09202/) (2022)
- [Learning Second Order Local Anomaly for General Face Forgery Detection](https://mlsanity.com/paper/2209.15490/) (2022)
- [Lips Are Lying: Spotting the Temporal Inconsistency between Audio and Visual in Lip-Syncing DeepFakes](https://mlsanity.com/paper/2401.15668/) (2024)
- [DiffusionFake: Enhancing Generalization in Deepfake Detection via Guided Stable Diffusion](https://mlsanity.com/paper/2410.04372/) (2024)
- [Contrastive Pseudo Learning for Open-World DeepFake Attribution](https://mlsanity.com/paper/2309.11132/) (2023)
- [DeepFake-Adapter: Dual-Level Adapter for DeepFake Detection](https://mlsanity.com/paper/2306.00863/) (2023)
- [Locate and Verify: A Two-Stream Network for Improved Deepfake Detection](https://mlsanity.com/paper/2309.11131/) (2023)

## Abstract (arXiv)

With the rapid development of facial manipulation techniques, face forgery detection has received considerable attention in digital media forensics due to security concerns. Most existing methods formulate face forgery detection as a classification problem and utilize binary labels or manipulated region masks as supervision. However, without considering the correlation between local regions, these global supervisions are insufficient to learn a generalized feature and prone to overfitting. To address this issue, we propose a novel perspective of face forgery detection via local relation learning. Specifically, we propose a Multi-scale Patch Similarity Module (MPSM), which measures the similarity between features of local regions and forms a robust and generalized similarity pattern. Moreover, we propose an RGB-Frequency Attention Module (RFAM) to fuse information in both RGB and frequency domains for more comprehensive local feature representation, which further improves the reliability of the similarity pattern. Extensive experiments show that the proposed method consistently outperforms the state-of-the-arts on widely-used benchmarks. Furthermore, detailed visualization shows the robustness and interpretability of our method.