# Frame-rate Up-conversion Detection Based on Convolutional Neural Network for Learning Spatiotemporal Features

arXiv: https://arxiv.org/abs/2103.13674 · HTML: https://mlsanity.com/paper/2103.13674/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 55 references that are in the map.
- First posted on arXiv 25 Mar 2021.

## Builds on (most cited first, 14 of 14)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [Network In Network](https://mlsanity.com/paper/1312.4400/) (2013)
- [Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation](https://mlsanity.com/paper/2010.16061/) (2020)
- [MesoNet: a Compact Facial Video Forgery Detection Network](https://mlsanity.com/paper/1809.00888/) (2018)
- [Media Forensics and DeepFakes: an overview](https://mlsanity.com/paper/2001.06564/) (2020)
- [FaceForensics: A Large-scale Video Dataset for Forgery Detection in Human Faces](https://mlsanity.com/paper/1803.09179/) (2018)
- [ForensicTransfer: Weakly-supervised Domain Adaptation for Forgery Detection](https://mlsanity.com/paper/1812.02510/) (2018)
- [Forensic Similarity for Digital Images](https://mlsanity.com/paper/1902.04684/) (2019)
- [Aligned and Non-Aligned Double JPEG Detection Using Convolutional Neural Networks](https://mlsanity.com/paper/1708.00930/) (2017)
- [Deep Convolutional Neural Network for Identifying Seam-Carving Forgery](https://mlsanity.com/paper/2007.02393/) (2020)
- [Block-level Double JPEG Compression Detection for Image Forgery Localization](https://mlsanity.com/paper/2003.09393/) (2020)
- [Motion-Compensated Coding and Frame-Rate Up-Conversion: Models and Analysis](https://mlsanity.com/paper/1404.3290/) (2014)

## Built on by (most cited first, 1 of 1)

- [DHNet: Double MPEG-4 Compression Detection via Multiple DCT Histograms](https://mlsanity.com/paper/2107.08939/) (2021)

## Abstract (arXiv)

With the advance in user-friendly and powerful video editing tools, anyone can easily manipulate videos without leaving prominent visual traces. Frame-rate up-conversion (FRUC), a representative temporal-domain operation, increases the motion continuity of videos with a lower frame-rate and is used by malicious counterfeiters in video tampering such as generating fake frame-rate video without improving the quality or mixing temporally spliced videos. FRUC is based on frame interpolation schemes and subtle artifacts that remain in interpolated frames are often difficult to distinguish. Hence, detecting such forgery traces is a critical issue in video forensics. This paper proposes a frame-rate conversion detection network (FCDNet) that learns forensic features caused by FRUC in an end-to-end fashion. The proposed network uses a stack of consecutive frames as the input and effectively learns interpolation artifacts using network blocks to learn spatiotemporal features. This study is the first attempt to apply a neural network to the detection of FRUC. Moreover, it can cover the following three types of frame interpolation schemes: nearest neighbor interpolation, bilinear interpolation, and motion-compensated interpolation. In contrast to existing methods that exploit all frames to verify integrity, the proposed approach achieves a high detection speed because it observes only six frames to test its authenticity. Extensive experiments were conducted with conventional forensic methods and neural networks for video forensic tasks to validate our research. The proposed network achieved state-of-the-art performance in terms of detecting the interpolated artifacts of FRUC. The experimental results also demonstrate that our trained model is robust for an unseen dataset, unlearned frame-rate, and unlearned quality factor.