# Spatiotemporal Inconsistency Learning for DeepFake Video Detection

arXiv: https://arxiv.org/abs/2109.01860 · HTML: https://mlsanity.com/paper/2109.01860/

## Key facts

- Cited by 88 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 37 references that are in the map.
- First posted on arXiv 4 Sep 2021.

## Builds on (most cited first, 29 of 29)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [FaceForensics++: Learning to Detect Manipulated Facial Images](https://mlsanity.com/paper/1901.08971/) (2019)
- [TSM: Temporal Shift Module for Efficient Video Understanding](https://mlsanity.com/paper/1811.08383/) (2018)
- [Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensics](https://mlsanity.com/paper/1909.12962/) (2019)
- [Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Clues](https://mlsanity.com/paper/2007.09355/) (2020)
- [Face X-ray for More General Face Forgery Detection](https://mlsanity.com/paper/1912.13458/) (2019)
- [Multi-attentional Deepfake Detection](https://mlsanity.com/paper/2103.02406/) (2021)
- [Exposing DeepFake Videos By Detecting Face Warping Artifacts](https://mlsanity.com/paper/1811.00656/) (2018)
- [Exposing Deep Fakes Using Inconsistent Head Poses](https://mlsanity.com/paper/1811.00661/) (2018)
- [On the Detection of Digital Face Manipulation](https://mlsanity.com/paper/1910.01717/) (2019)
- [FaceShifter: Towards High Fidelity And Occlusion Aware Face Swapping](https://mlsanity.com/paper/1912.13457/) (2019)
- [The Deepfake Detection Challenge (DFDC) Preview Dataset](https://mlsanity.com/paper/1910.08854/) (2019)
- [WildDeepfake: A Challenging Real-World Dataset for Deepfake Detection](https://mlsanity.com/paper/2101.01456/) (2021)
- [TDN: Temporal Difference Networks for Efficient Action Recognition](https://mlsanity.com/paper/2012.10071/) (2020)
- [Capsule-Forensics: Using Capsule Networks to Detect Forged Images and Videos](https://mlsanity.com/paper/1810.11215/) (2018)
- [Two-branch Recurrent Network for Isolating Deepfakes in Videos](https://mlsanity.com/paper/2008.03412/) (2020)
- [Multi-task Learning For Detecting and Segmenting Manipulated Facial Images and Videos](https://mlsanity.com/paper/1906.06876/) (2019)
- [Recurrent Convolutional Strategies for Face Manipulation Detection in Videos](https://mlsanity.com/paper/1905.00582/) (2019)
- [Local Relation Learning for Face Forgery Detection](https://mlsanity.com/paper/2105.02577/) (2021)
- [TEINet: Towards an Efficient Architecture for Video Recognition](https://mlsanity.com/paper/1911.09435/) (2019)
- [DeepRhythm: Exposing DeepFakes with Attentional Visual Heartbeat Rhythms](https://mlsanity.com/paper/2006.07634/) (2020)
- [Sharp Multiple Instance Learning for DeepFake Video Detection](https://mlsanity.com/paper/2008.04585/) (2020)
- [Learning Spatiotemporal Features with 3D Convolutional Networks](https://mlsanity.com/paper/1412.0767/) (2014)
- [Automatic Face Reenactment](https://mlsanity.com/paper/1602.02651/) (2016)
- [FakePolisher: Making DeepFakes More Detection-Evasive by Shallow Reconstruction](https://mlsanity.com/paper/2006.07533/) (2020)
- [Exposing Deep-faked Videos by Anomalous Co-motion Pattern Detection](https://mlsanity.com/paper/2008.04848/) (2020)

## Built on by (most cited first, 30 of 88)

- [Leveraging Real Talking Faces via Self-Supervision for Robust Forgery Detection](https://mlsanity.com/paper/2201.07131/) (2022)
- [Dual Contrastive Learning for General Face Forgery Detection](https://mlsanity.com/paper/2112.13522/) (2021)
- [TALL: Thumbnail Layout for Deepfake Video Detection](https://mlsanity.com/paper/2307.07494/) (2023)
- [DeMamba: AI-Generated Video Detection on Million-Scale GenVideo Benchmark](https://mlsanity.com/paper/2405.19707/) (2024)
- [AltFreezing for More General Video Face Forgery Detection](https://mlsanity.com/paper/2307.08317/) (2023)
- [Exploiting Fine-grained Face Forgery Clues via Progressive Enhancement Learning](https://mlsanity.com/paper/2112.13977/) (2021)
- [Exploiting Style Latent Flows for Generalizing Deepfake Video Detection](https://mlsanity.com/paper/2403.06592/) (2024)
- [Generalizing Deepfake Video Detection with Plug-and-Play: Video-Level Blending and Spatiotemporal Adapter Tuning](https://mlsanity.com/paper/2408.17065/) (2024)
- [Exploring Frequency Adversarial Attacks for Face Forgery Detection](https://mlsanity.com/paper/2203.15674/) (2022)
- [Glitch in the Matrix: A Large Scale Benchmark for Content Driven Audio-Visual Forgery Detection and Localization](https://mlsanity.com/paper/2305.01979/) (2023)
- [On Learning Multi-Modal Forgery Representation for Diffusion Generated Video Detection](https://mlsanity.com/paper/2410.23623/) (2024)
- [D3: Training-Free AI-Generated Video Detection Using Second-Order Features](https://mlsanity.com/paper/2508.00701/) (2025)
- [Parents and Children: Distinguishing Multimodal DeepFakes from Natural Images](https://mlsanity.com/paper/2304.00500/) (2023)
- [Contrastive Pseudo Learning for Open-World DeepFake Attribution](https://mlsanity.com/paper/2309.11132/) (2023)
- [Exploring Unbiased Deepfake Detection via Token-Level Shuffling and Mixing](https://mlsanity.com/paper/2501.04376/) (2025)
- [DeepFake-Adapter: Dual-Level Adapter for DeepFake Detection](https://mlsanity.com/paper/2306.00863/) (2023)
- [MINTIME: Multi-Identity Size-Invariant Video Deepfake Detection](https://mlsanity.com/paper/2211.10996/) (2022)
- [Towards a Universal Synthetic Video Detector: From Face or Background Manipulations to Fully AI-Generated Content](https://mlsanity.com/paper/2412.12278/) (2024)
- [Learning Spatiotemporal Inconsistency via Thumbnail Layout for Face Deepfake Detection](https://mlsanity.com/paper/2403.10261/) (2024)
- [Digital and Physical Face Attacks: Reviewing and One Step Further](https://mlsanity.com/paper/2209.14692/) (2022)
- [Learning Human-Perceived Fakeness in AI-Generated Videos via Multimodal LLMs](https://mlsanity.com/paper/2509.22646/) (2025)
- [SoK: Systematization and Benchmarking of Deepfake Detectors in a Unified Framework](https://mlsanity.com/paper/2401.04364/) (2024)
- [GenConViT: Deepfake Video Detection Using Generative Convolutional Vision Transformer](https://mlsanity.com/paper/2307.07036/) (2023)
- [A Survey of Defenses Against AI-Generated Visual Media: Detection,Disruption, and Authentication](https://mlsanity.com/paper/2407.10575/) (2024)
- [FedForgery: Generalized Face Forgery Detection with Residual Federated Learning](https://mlsanity.com/paper/2210.09563/) (2022)
- [Vulnerability-Aware Spatio-Temporal Learning for Generalizable Deepfake Video Detection](https://mlsanity.com/paper/2501.01184/) (2025)
- [FakeOut: Leveraging Out-of-domain Self-supervision for Multi-modal Video Deepfake Detection](https://mlsanity.com/paper/2212.00773/) (2022)
- [GenWorld: Towards Detecting AI-generated Real-world Simulation Videos](https://mlsanity.com/paper/2506.10975/) (2025)
- [An adversarial attack approach for eXplainable AI evaluation on deepfake detection models](https://mlsanity.com/paper/2312.06627/) (2023)
- [DeepShield: Fortifying Deepfake Video Detection with Local and Global Forgery Analysis](https://mlsanity.com/paper/2510.25237/) (2025)

## Abstract (arXiv)

The rapid development of facial manipulation techniques has aroused public concerns in recent years. Following the success of deep learning, existing methods always formulate DeepFake video detection as a binary classification problem and develop frame-based and video-based solutions. However, little attention has been paid to capturing the spatial-temporal inconsistency in forged videos. To address this issue, we term this task as a Spatial-Temporal Inconsistency Learning (STIL) process and instantiate it into a novel STIL block, which consists of a Spatial Inconsistency Module (SIM), a Temporal Inconsistency Module (TIM), and an Information Supplement Module (ISM). Specifically, we present a novel temporal modeling paradigm in TIM by exploiting the temporal difference over adjacent frames along with both horizontal and vertical directions. And the ISM simultaneously utilizes the spatial information from SIM and temporal information from TIM to establish a more comprehensive spatial-temporal representation. Moreover, our STIL block is flexible and could be plugged into existing 2D CNNs. Extensive experiments and visualizations are presented to demonstrate the effectiveness of our method against the state-of-the-art competitors.