# A Novel Framework for Assessment of Learning-based Detectors in Realistic Conditions with Application to Deepfake Detection

arXiv: https://arxiv.org/abs/2203.11797 · HTML: https://mlsanity.com/paper/2203.11797/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 19 of its 21 references that are in the map.
- First posted on arXiv 22 Mar 2022.

## Builds on (most cited first, 19 of 19)

- [Benchmarking Neural Network Robustness to Common Corruptions and Perturbations](https://mlsanity.com/paper/1903.12261/) (2019)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [FaceForensics++: Learning to Detect Manipulated Facial Images](https://mlsanity.com/paper/1901.08971/) (2019)
- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensics](https://mlsanity.com/paper/1909.12962/) (2019)
- [Face X-ray for More General Face Forgery Detection](https://mlsanity.com/paper/1912.13458/) (2019)
- [MesoNet: a Compact Facial Video Forgery Detection Network](https://mlsanity.com/paper/1809.00888/) (2018)
- [Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Coming](https://mlsanity.com/paper/1907.07484/) (2019)
- [DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detection](https://mlsanity.com/paper/2001.03024/) (2020)
- [Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domain](https://mlsanity.com/paper/2103.01856/) (2021)
- [The DeepFake Detection Challenge (DFDC) Dataset](https://mlsanity.com/paper/2006.07397/) (2020)
- [Two-Stream Neural Networks for Tampered Face Detection](https://mlsanity.com/paper/1803.11276/) (2018)
- [Understanding How Image Quality Affects Deep Neural Networks](https://mlsanity.com/paper/1604.04004/) (2016)
- [Adversarial Examples Are a Natural Consequence of Test Error in Noise](https://mlsanity.com/paper/1901.10513/) (2019)
- [Benchmarking the Robustness of Semantic Segmentation Models](https://mlsanity.com/paper/1908.05005/) (2019)
- [Use of a Capsule Network to Detect Fake Images and Videos](https://mlsanity.com/paper/1910.12467/) (2019)
- [On Low-Resolution Face Recognition in the Wild: Comparisons and New Techniques](https://mlsanity.com/paper/1805.11529/) (2018)
- [A simple way to make neural networks robust against diverse image corruptions](https://mlsanity.com/paper/2001.06057/) (2020)
- [On Classification of Distorted Images with Deep Convolutional Neural Networks](https://mlsanity.com/paper/1701.01924/) (2017)

## Abstract (arXiv)

Deep convolutional neural networks have shown remarkable results on multiple detection tasks. Despite the significant progress, the performance of such detectors are often assessed in public benchmarks under non-realistic conditions. Specifically, impact of conventional distortions and processing operations such as compression, noise, and enhancement are not sufficiently studied. This paper proposes a rigorous framework to assess performance of learning-based detectors in more realistic situations. An illustrative example is shown under deepfake detection context. Inspired by the assessment results, a data augmentation strategy based on natural image degradation process is designed, which significantly improves the generalization ability of two deepfake detectors.