# Recasting Residual-based Local Descriptors as Convolutional Neural Networks: an Application to Image Forgery Detection

arXiv: https://arxiv.org/abs/1703.04615 · HTML: https://mlsanity.com/paper/1703.04615/

## Key facts

- Cited by 56 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Mar 2017.

## Built on by (most cited first, 30 of 56)

- [Towards Universal Fake Image Detectors that Generalize Across Generative Models](https://mlsanity.com/paper/2302.10174/) (2023)
- [Multi-attentional Deepfake Detection](https://mlsanity.com/paper/2103.02406/) (2021)
- [Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domain](https://mlsanity.com/paper/2103.01856/) (2021)
- [Frequency-aware Discriminative Feature Learning Supervised by Single-Center Loss for Face Forgery Detection](https://mlsanity.com/paper/2103.09096/) (2021)
- [Leveraging Real Talking Faces via Self-Supervision for Robust Forgery Detection](https://mlsanity.com/paper/2201.07131/) (2022)
- [Implicit Identity Leakage: The Stumbling Block to Improving Deepfake Detection Generalization](https://mlsanity.com/paper/2210.14457/) (2022)
- [M2TR: Multi-modal Multi-scale Transformers for Deepfake Detection](https://mlsanity.com/paper/2104.09770/) (2021)
- [Local Relation Learning for Face Forgery Detection](https://mlsanity.com/paper/2105.02577/) (2021)
- [Robustness of AI-Image Detectors: Fundamental Limits and Practical Attacks](https://mlsanity.com/paper/2310.00076/) (2023)
- [CORE: Consistent Representation Learning for Face Forgery Detection](https://mlsanity.com/paper/2206.02749/) (2022)
- [Exploiting Fine-grained Face Forgery Clues via Progressive Enhancement Learning](https://mlsanity.com/paper/2112.13977/) (2021)
- [MARLIN: Masked Autoencoder for facial video Representation LearnINg](https://mlsanity.com/paper/2211.06627/) (2022)
- [SeeABLE: Soft Discrepancies and Bounded Contrastive Learning for Exposing Deepfakes](https://mlsanity.com/paper/2211.11296/) (2022)
- [Exploring Frequency Adversarial Attacks for Face Forgery Detection](https://mlsanity.com/paper/2203.15674/) (2022)
- [Glitch in the Matrix: A Large Scale Benchmark for Content Driven Audio-Visual Forgery Detection and Localization](https://mlsanity.com/paper/2305.01979/) (2023)
- [Perceptual Artifacts Localization for Image Synthesis Tasks](https://mlsanity.com/paper/2310.05590/) (2023)
- [Detect and Locate: Exposing Face Manipulation by Semantic- and Noise-level Telltales](https://mlsanity.com/paper/2107.05821/) (2021)
- [Exploring Disentangled Content Information for Face Forgery Detection](https://mlsanity.com/paper/2207.09202/) (2022)
- [Explaining Deepfake Detection by Analysing Image Matching](https://mlsanity.com/paper/2207.09679/) (2022)
- [Wavelet-Packets for Deepfake Image Analysis and Detection](https://mlsanity.com/paper/2106.09369/) (2021)
- [GLFF: Global and Local Feature Fusion for AI-synthesized Image Detection](https://mlsanity.com/paper/2211.08615/) (2022)
- [TAR: Generalized Forensic Framework to Detect Deepfakes using Weakly Supervised Learning](https://mlsanity.com/paper/2105.06117/) (2021)
- [FaceGuard: Proactive Deepfake Detection](https://mlsanity.com/paper/2109.05673/) (2021)
- [Detection and Localization of Facial Expression Manipulations](https://mlsanity.com/paper/2103.08134/) (2021)
- [Deepfake Forensics via An Adversarial Game](https://mlsanity.com/paper/2103.13567/) (2021)
- [Finding Facial Forgery Artifacts with Parts-Based Detectors](https://mlsanity.com/paper/2109.10688/) (2021)
- [Learning Pairwise Interaction for Generalizable DeepFake Detection](https://mlsanity.com/paper/2302.13288/) (2023)
- [Metric Learning for Anti-Compression Facial Forgery Detection](https://mlsanity.com/paper/2103.08397/) (2021)
- [Detection of Deepfake Videos Using Long Distance Attention](https://mlsanity.com/paper/2106.12832/) (2021)
- [DeePhy: On Deepfake Phylogeny](https://mlsanity.com/paper/2209.09111/) (2022)

## Abstract (arXiv)

Local descriptors based on the image noise residual have proven extremely effective for a number of forensic applications, like forgery detection and localization. Nonetheless, motivated by promising results in computer vision, the focus of the research community is now shifting on deep learning. In this paper we show that a class of residual-based descriptors can be actually regarded as a simple constrained convolutional neural network (CNN). Then, by relaxing the constraints, and fine-tuning the net on a relatively small training set, we obtain a significant performance improvement with respect to the conventional detector.