# Identifying Rhythmic Patterns for Face Forgery Detection and Categorization

arXiv: https://arxiv.org/abs/2207.01199 · HTML: https://mlsanity.com/paper/2207.01199/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 30 references that are in the map.
- First posted on arXiv 4 Jul 2022.

## Builds on (most cited first, 20 of 20)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Progressive Growing of GANs for Improved Quality, Stability, and Variation](https://mlsanity.com/paper/1710.10196/) (2017)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [FaceForensics++: Learning to Detect Manipulated Facial Images](https://mlsanity.com/paper/1901.08971/) (2019)
- [StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation](https://mlsanity.com/paper/1711.09020/) (2017)
- [Face2Face: Real-time Face Capture and Reenactment of RGB Videos](https://mlsanity.com/paper/2007.14808/) (2020)
- [Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensics](https://mlsanity.com/paper/1909.12962/) (2019)
- [Face X-ray for More General Face Forgery Detection](https://mlsanity.com/paper/1912.13458/) (2019)
- [Exposing DeepFake Videos By Detecting Face Warping Artifacts](https://mlsanity.com/paper/1811.00656/) (2018)
- [Deferred Neural Rendering: Image Synthesis using Neural Textures](https://mlsanity.com/paper/1904.12356/) (2019)
- [Global Texture Enhancement for Fake Face Detection in the Wild](https://mlsanity.com/paper/2002.00133/) (2020)
- [FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals](https://mlsanity.com/paper/1901.02212/) (2019)
- [RhythmNet: End-to-end Heart Rate Estimation from Face via Spatial-temporal Representation](https://mlsanity.com/paper/1910.11515/) (2019)
- [Remote Heart Rate Measurement from Highly Compressed Facial Videos: an End-to-end Deep Learning Solution with Video Enhancement](https://mlsanity.com/paper/1907.11921/) (2019)
- [Video-based Remote Physiological Measurement via Cross-verified Feature Disentangling](https://mlsanity.com/paper/2007.08213/) (2020)
- [DeepRhythm: Exposing DeepFakes with Attentional Visual Heartbeat Rhythms](https://mlsanity.com/paper/2006.07634/) (2020)
- [How Do the Hearts of Deep Fakes Beat? Deep Fake Source Detection via Interpreting Residuals with Biological Signals](https://mlsanity.com/paper/2008.11363/) (2020)
- [Swapped Face Detection using Deep Learning and Subjective Assessment](https://mlsanity.com/paper/1909.04217/) (2019)

## Built on by (most cited first, 3 of 3)

- [Exploring Disentangled Content Information for Face Forgery Detection](https://mlsanity.com/paper/2207.09202/) (2022)
- [Diff-ID: An Explainable Identity Difference Quantification Framework for DeepFake Detection](https://mlsanity.com/paper/2303.18174/) (2023)
- [Deep Learning Technology for Face Forgery Detection: A Survey](https://mlsanity.com/paper/2409.14289/) (2024)

## Abstract (arXiv)

With the emergence of GAN, face forgery technologies have been heavily abused. Achieving accurate face forgery detection is imminent. Inspired by remote photoplethysmography (rPPG) that PPG signal corresponds to the periodic change of skin color caused by heartbeat in face videos, we observe that despite the inevitable loss of PPG signal during the forgery process, there is still a mixture of PPG signals in the forgery video with a unique rhythmic pattern depending on its generation method. Motivated by this key observation, we propose a framework for face forgery detection and categorization consisting of: 1) a Spatial-Temporal Filtering Network (STFNet) for PPG signals filtering, and 2) a Spatial-Temporal Interaction Network (STINet) for constraint and interaction of PPG signals. Moreover, with insight into the generation of forgery methods, we further propose intra-source and inter-source blending to boost the performance of the framework. Overall, extensive experiments have proved the superiority of our method.