# Detect and Locate: Exposing Face Manipulation by Semantic- and Noise-level Telltales

arXiv: https://arxiv.org/abs/2107.05821 · HTML: https://mlsanity.com/paper/2107.05821/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 63 references that are in the map.
- First posted on arXiv 13 Jul 2021.

## Builds on (most cited first, 30 of 39)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [FaceForensics++: Learning to Detect Manipulated Facial Images](https://mlsanity.com/paper/1901.08971/) (2019)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [Face2Face: Real-time Face Capture and Reenactment of RGB Videos](https://mlsanity.com/paper/2007.14808/) (2020)
- [Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensics](https://mlsanity.com/paper/1909.12962/) (2019)
- [Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Clues](https://mlsanity.com/paper/2007.09355/) (2020)
- [Face X-ray for More General Face Forgery Detection](https://mlsanity.com/paper/1912.13458/) (2019)
- [MesoNet: a Compact Facial Video Forgery Detection Network](https://mlsanity.com/paper/1809.00888/) (2018)
- [Learning Deconvolution Network for Semantic Segmentation](https://mlsanity.com/paper/1505.04366/) (2015)
- [Multi-attentional Deepfake Detection](https://mlsanity.com/paper/2103.02406/) (2021)
- [Exposing Deep Fakes Using Inconsistent Head Poses](https://mlsanity.com/paper/1811.00661/) (2018)
- [On the Detection of Digital Face Manipulation](https://mlsanity.com/paper/1910.01717/) (2019)
- [Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domain](https://mlsanity.com/paper/2103.01856/) (2021)
- [AttGAN: Facial Attribute Editing by Only Changing What You Want](https://mlsanity.com/paper/1711.10678/) (2017)
- [DeepFakes: a New Threat to Face Recognition? Assessment and Detection](https://mlsanity.com/paper/1812.08685/) (2018)
- [Media Forensics and DeepFakes: an overview](https://mlsanity.com/paper/2001.06564/) (2020)
- [Two-branch Recurrent Network for Isolating Deepfakes in Videos](https://mlsanity.com/paper/2008.03412/) (2020)
- [Learning Rich Features for Image Manipulation Detection](https://mlsanity.com/paper/1805.04953/) (2018)
- [The DeepFake Detection Challenge (DFDC) Dataset](https://mlsanity.com/paper/2006.07397/) (2020)
- [Multi-task Learning For Detecting and Segmenting Manipulated Facial Images and Videos](https://mlsanity.com/paper/1906.06876/) (2019)
- [Noiseprint: a CNN-based camera model fingerprint](https://mlsanity.com/paper/1808.08396/) (2018)
- [Learning to Compare Image Patches via Convolutional Neural Networks](https://mlsanity.com/paper/1504.03641/) (2015)

## Built on by (most cited first, 30 of 39)

- [Beyond the Prior Forgery Knowledge: Mining Critical Clues for General Face Forgery Detection](https://mlsanity.com/paper/2304.12489/) (2023)
- [AutoSplice: A Text-prompt Manipulated Image Dataset for Media Forensics](https://mlsanity.com/paper/2304.06870/) (2023)
- [FakeBench: Probing Explainable Fake Image Detection via Large Multimodal Models](https://mlsanity.com/paper/2404.13306/) (2024)
- [OmniGuard: Hybrid Manipulation Localization via Augmented Versatile Deep Image Watermarking](https://mlsanity.com/paper/2412.01615/) (2024)
- [Pixel-Inconsistency Modeling for Image Manipulation Localization](https://mlsanity.com/paper/2310.00234/) (2023)
- [MoE-FFD: Mixture of Experts for Generalized and Parameter-Efficient Face Forgery Detection](https://mlsanity.com/paper/2404.08452/) (2024)
- [Detect Any Deepfakes: Segment Anything Meets Face Forgery Detection and Localization](https://mlsanity.com/paper/2306.17075/) (2023)
- [Digital and Physical Face Attacks: Reviewing and One Step Further](https://mlsanity.com/paper/2209.14692/) (2022)
- [Enhancing General Face Forgery Detection via Vision Transformer with Low-Rank Adaptation](https://mlsanity.com/paper/2303.00917/) (2023)
- [The Age of Synthetic Realities: Challenges and Opportunities](https://mlsanity.com/paper/2306.11503/) (2023)
- [Mitigating the Curse of Dimensionality for Certified Robustness via Dual Randomized Smoothing](https://mlsanity.com/paper/2404.09586/) (2024)
- [Delocate: Detection and Localization for Deepfake Videos with Randomly-Located Tampered Traces](https://mlsanity.com/paper/2401.13516/) (2024)
- [Multi-spectral Class Center Network for Face Manipulation Detection and Localization](https://mlsanity.com/paper/2305.10794/) (2023)
- [Mixture-of-Noises Enhanced Forgery-Aware Predictor for Multi-Face Manipulation Detection and Localization](https://mlsanity.com/paper/2408.02306/) (2024)
- [Open-Set Deepfake Detection: A Parameter-Efficient Adaptation Method with Forgery Style Mixture](https://mlsanity.com/paper/2408.12791/) (2024)
- [Fair Deepfake Detectors Can Generalize](https://mlsanity.com/paper/2507.02645/) (2025)
- [Two-branch Multi-scale Deep Neural Network for Generalized Document Recapture Attack Detection](https://mlsanity.com/paper/2211.16786/) (2022)
- [ForensicsSAM: Toward Robust and Unified Image Forgery Detection and Localization Resisting to Adversarial Attack](https://mlsanity.com/paper/2508.07402/) (2025)
- [Segment Anything for Videos: A Systematic Survey](https://mlsanity.com/paper/2408.08315/) (2024)
- [NPVForensics: Jointing Non-critical Phonemes and Visemes for Deepfake Detection](https://mlsanity.com/paper/2306.06885/) (2023)
- [A Quality-Centric Framework for Generic Deepfake Detection](https://mlsanity.com/paper/2411.05335/) (2024)
- [Passive Deepfake Detection Across Multi-modalities: A Comprehensive Survey](https://mlsanity.com/paper/2411.17911/) (2024)
- [FractalForensics: Proactive Deepfake Detection and Localization via Fractal Watermarks](https://mlsanity.com/paper/2504.09451/) (2025)
- [DDL: A Large-Scale Dataset for Deepfake Detection and Localization in Diversified Real-World Scenarios](https://mlsanity.com/paper/2506.23292/) (2025)
- [Pleno-Generation: A Scalable Generative Face Video Compression Framework with Bandwidth Intelligence](https://mlsanity.com/paper/2502.17085/) (2025)
- [Fighting Malicious Media Data: A Survey on Tampering Detection and Deepfake Detection](https://mlsanity.com/paper/2212.05667/) (2022)
- [Robust face anti-spoofing framework with Convolutional Vision Transformer](https://mlsanity.com/paper/2307.12459/) (2023)
- [Propose and Rectify: A Forensics-Driven MLLM Framework for Image Manipulation Localization](https://mlsanity.com/paper/2508.17976/) (2025)
- [RecoverMark: Robust Watermarking for Localization and Recovery of Manipulated Faces](https://mlsanity.com/paper/2602.20618/) (2026)
- [Deepfake Detection via Joint Unsupervised Reconstruction and Supervised Classification](https://mlsanity.com/paper/2211.13424/) (2022)

## Abstract (arXiv)

The technological advancements of deep learning have enabled sophisticated face manipulation schemes, raising severe trust issues and security concerns in modern society. Generally speaking, detecting manipulated faces and locating the potentially altered regions are challenging tasks. Herein, we propose a conceptually simple but effective method to efficiently detect forged faces in an image while simultaneously locating the manipulated regions. The proposed scheme relies on a segmentation map that delivers meaningful high-level semantic information clues about the image. Furthermore, a noise map is estimated, playing a complementary role in capturing low-level clues and subsequently empowering decision-making. Finally, the features from these two modules are combined to distinguish fake faces. Extensive experiments show that the proposed model achieves state-of-the-art detection accuracy and remarkable localization performance.