# Visual Representations of Physiological Signals for Fake Video Detection

arXiv: https://arxiv.org/abs/2207.08380 · HTML: https://mlsanity.com/paper/2207.08380/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 36 references that are in the map.
- First posted on arXiv 18 Jul 2022.

## Builds on (most cited first, 21 of 21)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [FaceForensics++: Learning to Detect Manipulated Facial Images](https://mlsanity.com/paper/1901.08971/) (2019)
- [MesoNet: a Compact Facial Video Forgery Detection Network](https://mlsanity.com/paper/1809.00888/) (2018)
- [Exposing DeepFake Videos By Detecting Face Warping Artifacts](https://mlsanity.com/paper/1811.00656/) (2018)
- [Exposing Deep Fakes Using Inconsistent Head Poses](https://mlsanity.com/paper/1811.00661/) (2018)
- [The Creation and Detection of Deepfakes: A Survey](https://mlsanity.com/paper/2004.11138/) (2020)
- [DeepFakes: a New Threat to Face Recognition? Assessment and Detection](https://mlsanity.com/paper/1812.08685/) (2018)
- [The Deepfake Detection Challenge (DFDC) Preview Dataset](https://mlsanity.com/paper/1910.08854/) (2019)
- [Capsule-Forensics: Using Capsule Networks to Detect Forged Images and Videos](https://mlsanity.com/paper/1810.11215/) (2018)
- [FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals](https://mlsanity.com/paper/1901.02212/) (2019)
- [S3FD: Single Shot Scale-invariant Face Detector](https://mlsanity.com/paper/1708.05237/) (2017)
- [Multi-Task Temporal Shift Attention Networks for On-Device Contactless Vitals Measurement](https://mlsanity.com/paper/2006.03790/) (2020)
- [The DeepFake Detection Challenge (DFDC) Dataset](https://mlsanity.com/paper/2006.07397/) (2020)
- [Multi-task Learning For Detecting and Segmenting Manipulated Facial Images and Videos](https://mlsanity.com/paper/1906.06876/) (2019)
- [Two-Stream Neural Networks for Tampered Face Detection](https://mlsanity.com/paper/1803.11276/) (2018)
- [Emotions Don't Lie: An Audio-Visual Deepfake Detection Method Using Affective Cues](https://mlsanity.com/paper/2003.06711/) (2020)
- [Not made for each other- Audio-Visual Dissonance-based Deepfake Detection and Localization](https://mlsanity.com/paper/2005.14405/) (2020)
- [Unmasking DeepFakes with simple Features](https://mlsanity.com/paper/1911.00686/) (2019)
- [Swapped Face Detection using Deep Learning and Subjective Assessment](https://mlsanity.com/paper/1909.04217/) (2019)

## Built on by (most cited first, 2 of 2)

- [DPL: Cross-quality DeepFake Detection via Dual Progressive Learning](https://mlsanity.com/paper/2410.07633/) (2024)
- [Detecting AI-Generated Video: A Vision-Language Dual-View Survey](https://mlsanity.com/paper/2607.10787/) (2026)

## Abstract (arXiv)

Realistic fake videos are a potential tool for spreading harmful misinformation given our increasing online presence and information intake. This paper presents a multimodal learning-based method for detection of real and fake videos. The method combines information from three modalities - audio, video, and physiology. We investigate two strategies for combining the video and physiology modalities, either by augmenting the video with information from the physiology or by novelly learning the fusion of those two modalities with a proposed Graph Convolutional Network architecture. Both strategies for combining the two modalities rely on a novel method for generation of visual representations of physiological signals. The detection of real and fake videos is then based on the dissimilarity between the audio and modified video modalities. The proposed method is evaluated on two benchmark datasets and the results show significant increase in detection performance compared to previous methods.