# Efficient Temporally-Aware DeepFake Detection using H.264 Motion Vectors

arXiv: https://arxiv.org/abs/2311.10788 · HTML: https://mlsanity.com/paper/2311.10788/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 46 references that are in the map.
- First posted on arXiv 17 Nov 2023.

## Builds on (most cited first, 21 of 21)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [RAFT: Recurrent All-Pairs Field Transforms for Optical Flow](https://mlsanity.com/paper/2003.12039/) (2020)
- [Searching for MobileNetV3](https://mlsanity.com/paper/1905.02244/) (2019)
- [FaceForensics++: Learning to Detect Manipulated Facial Images](https://mlsanity.com/paper/1901.08971/) (2019)
- [Two-Stream Convolutional Networks for Action Recognition in Videos](https://mlsanity.com/paper/1406.2199/) (2014)
- [Face2Face: Real-time Face Capture and Reenactment of RGB Videos](https://mlsanity.com/paper/2007.14808/) (2020)
- [Albumentations: fast and flexible image augmentations](https://mlsanity.com/paper/1809.06839/) (2018)
- [Face X-ray for More General Face Forgery Detection](https://mlsanity.com/paper/1912.13458/) (2019)
- [Detecting Deepfakes with Self-Blended Images](https://mlsanity.com/paper/2204.08376/) (2022)
- [Media Forensics and DeepFakes: an overview](https://mlsanity.com/paper/2001.06564/) (2020)
- [E-RAFT: Dense Optical Flow from Event Cameras](https://mlsanity.com/paper/2108.10552/) (2021)
- [FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals](https://mlsanity.com/paper/1901.02212/) (2019)
- [GridMask Data Augmentation](https://mlsanity.com/paper/2001.04086/) (2020)
- [FaceForensics: A Large-scale Video Dataset for Forgery Detection in Human Faces](https://mlsanity.com/paper/1803.09179/) (2018)
- [Compressed Video Action Recognition](https://mlsanity.com/paper/1712.00636/) (2017)
- [Fast Optical Flow using Dense Inverse Search](https://mlsanity.com/paper/1603.03590/) (2016)
- [DeepRhythm: Exposing DeepFakes with Attentional Visual Heartbeat Rhythms](https://mlsanity.com/paper/2006.07634/) (2020)
- [Recasting Residual-based Local Descriptors as Convolutional Neural Networks: an Application to Image Forgery Detection](https://mlsanity.com/paper/1703.04615/) (2017)
- [How Do the Hearts of Deep Fakes Beat? Deep Fake Source Detection via Interpreting Residuals with Biological Signals](https://mlsanity.com/paper/2008.11363/) (2020)
- [Deepfake Video Detection with Spatiotemporal Dropout Transformer](https://mlsanity.com/paper/2207.06612/) (2022)

## Built on by (most cited first, 3 of 3)

- [Zero-Shot Visual Deepfake Detection: Can AI Predict and Prevent Fake Content Before It's Created?](https://mlsanity.com/paper/2509.18461/) (2025)
- [Detect Early, Escalate Rarely: Anytime Detection of AI-Generated Video from the Compressed Bitstream](https://mlsanity.com/paper/2607.19476/) (2026)
- [MVTrack: Ultrafast Appearance-Free Moving Object Tracking from Compressed Bitstreams](https://mlsanity.com/paper/2608.10790/) (2026)

## Abstract (arXiv)

Video DeepFakes are fake media created with Deep Learning (DL) that manipulate a person's expression or identity. Most current DeepFake detection methods analyze each frame independently, ignoring inconsistencies and unnatural movements between frames. Some newer methods employ optical flow models to capture this temporal aspect, but they are computationally expensive. In contrast, we propose using the related but often ignored Motion Vectors (MVs) and Information Masks (IMs) from the H.264 video codec, to detect temporal inconsistencies in DeepFakes. Our experiments show that this approach is effective and has minimal computational costs, compared with per-frame RGB-only methods. This could lead to new, real-time temporally-aware DeepFake detection methods for video calls and streaming.