# AltFreezing for More General Video Face Forgery Detection

arXiv: https://arxiv.org/abs/2307.08317 · HTML: https://mlsanity.com/paper/2307.08317/

## Key facts

- Cited by 78 papers in the map (data through 14 Oct 2026).
- Builds on 41 of its 57 references that are in the map.
- First posted on arXiv 17 Jul 2023.

## Builds on (most cited first, 30 of 41)

- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [Learning Deep Features for Discriminative Localization](https://mlsanity.com/paper/1512.04150/) (2015)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [FaceForensics++: Learning to Detect Manipulated Facial Images](https://mlsanity.com/paper/1901.08971/) (2019)
- [CNN-generated images are surprisingly easy to spot... for now](https://mlsanity.com/paper/1912.11035/) (2019)
- [Face2Face: Real-time Face Capture and Reenactment of RGB Videos](https://mlsanity.com/paper/2007.14808/) (2020)
- [Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensics](https://mlsanity.com/paper/1909.12962/) (2019)
- [Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Clues](https://mlsanity.com/paper/2007.09355/) (2020)
- [Face X-ray for More General Face Forgery Detection](https://mlsanity.com/paper/1912.13458/) (2019)
- [MesoNet: a Compact Facial Video Forgery Detection Network](https://mlsanity.com/paper/1809.00888/) (2018)
- [Detecting Deepfakes with Self-Blended Images](https://mlsanity.com/paper/2204.08376/) (2022)
- [Exposing DeepFake Videos By Detecting Face Warping Artifacts](https://mlsanity.com/paper/1811.00656/) (2018)
- [Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detection](https://mlsanity.com/paper/2012.07657/) (2020)
- [What makes fake images detectable? Understanding properties that generalize](https://mlsanity.com/paper/2008.10588/) (2020)
- [On the Detection of Digital Face Manipulation](https://mlsanity.com/paper/1910.01717/) (2019)
- [DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detection](https://mlsanity.com/paper/2001.03024/) (2020)
- [Deferred Neural Rendering: Image Synthesis using Neural Textures](https://mlsanity.com/paper/1904.12356/) (2019)
- [Neural Voice Puppetry: Audio-driven Facial Reenactment](https://mlsanity.com/paper/1912.05566/) (2019)
- [Talking Face Generation by Adversarially Disentangled Audio-Visual Representation](https://mlsanity.com/paper/1807.07860/) (2018)
- [Learning Self-Consistency for Deepfake Detection](https://mlsanity.com/paper/2012.09311/) (2020)
- [Two-branch Recurrent Network for Isolating Deepfakes in Videos](https://mlsanity.com/paper/2008.03412/) (2020)
- [Exploring Temporal Coherence for More General Video Face Forgery Detection](https://mlsanity.com/paper/2108.06693/) (2021)
- [Realistic Speech-Driven Facial Animation with GANs](https://mlsanity.com/paper/1906.06337/) (2019)
- [Multi-task Learning For Detecting and Segmenting Manipulated Facial Images and Videos](https://mlsanity.com/paper/1906.06876/) (2019)
- [Leveraging Real Talking Faces via Self-Supervision for Robust Forgery Detection](https://mlsanity.com/paper/2201.07131/) (2022)
- [Recurrent Convolutional Strategies for Face Manipulation Detection in Videos](https://mlsanity.com/paper/1905.00582/) (2019)
- [Emotions Don't Lie: An Audio-Visual Deepfake Detection Method Using Affective Cues](https://mlsanity.com/paper/2003.06711/) (2020)
- [One Shot Face Swapping on Megapixels](https://mlsanity.com/paper/2105.04932/) (2021)
- [ID-Reveal: Identity-aware DeepFake Video Detection](https://mlsanity.com/paper/2012.02512/) (2020)
- [Protecting Celebrities from DeepFake with Identity Consistency Transformer](https://mlsanity.com/paper/2203.01318/) (2022)

## Built on by (most cited first, 30 of 78)

- [Transcending Forgery Specificity with Latent Space Augmentation for Generalizable Deepfake Detection](https://mlsanity.com/paper/2311.11278/) (2023)
- [LAA-Net: Localized Artifact Attention Network for Quality-Agnostic and Generalizable Deepfake Detection](https://mlsanity.com/paper/2401.13856/) (2024)
- [Exploiting Style Latent Flows for Generalizing Deepfake Video Detection](https://mlsanity.com/paper/2403.06592/) (2024)
- [AVFF: Audio-Visual Feature Fusion for Video Deepfake Detection](https://mlsanity.com/paper/2406.02951/) (2024)
- [Generalizing Deepfake Video Detection with Plug-and-Play: Video-Level Blending and Spatiotemporal Adapter Tuning](https://mlsanity.com/paper/2408.17065/) (2024)
- [Can We Leave Deepfake Data Behind in Training Deepfake Detector?](https://mlsanity.com/paper/2408.17052/) (2024)
- [Diffusion Facial Forgery Detection](https://mlsanity.com/paper/2401.15859/) (2024)
- [On Learning Multi-Modal Forgery Representation for Diffusion Generated Video Detection](https://mlsanity.com/paper/2410.23623/) (2024)
- [Rethinking Vision-Language Model in Face Forensics: Multi-Modal Interpretable Forged Face Detector](https://mlsanity.com/paper/2503.20188/) (2025)
- [Towards More General Video-based Deepfake Detection through Facial Component Guided Adaptation for Foundation Model](https://mlsanity.com/paper/2404.05583/) (2024)
- [X2-DFD: A framework for eXplainable and eXtendable Deepfake Detection](https://mlsanity.com/paper/2410.06126/) (2024)
- [Learning Natural Consistency Representation for Face Forgery Video Detection](https://mlsanity.com/paper/2407.10550/) (2024)
- [FSFM: A Generalizable Face Security Foundation Model via Self-Supervised Facial Representation Learning](https://mlsanity.com/paper/2412.12032/) (2024)
- [Seeing What Matters: Generalizable AI-generated Video Detection with Forensic-Oriented Augmentation](https://mlsanity.com/paper/2506.16802/) (2025)
- [Deepfake Detection that Generalizes Across Benchmarks](https://mlsanity.com/paper/2508.06248/) (2025)
- [Robust Identity Perceptual Watermark Against Deepfake Face Swapping](https://mlsanity.com/paper/2311.01357/) (2023)
- [Unified Frequency-Assisted Transformer Framework for Detecting and Grounding Multi-Modal Manipulation](https://mlsanity.com/paper/2309.09667/) (2023)
- [SoK: Systematization and Benchmarking of Deepfake Detectors in a Unified Framework](https://mlsanity.com/paper/2401.04364/) (2024)
- [From Specificity to Generality: Revisiting Generalizable Artifacts in Detecting Face Deepfakes](https://mlsanity.com/paper/2504.04827/) (2025)
- [A Survey of Defenses Against AI-Generated Visual Media: Detection,Disruption, and Authentication](https://mlsanity.com/paper/2407.10575/) (2024)
- [Beyond Spatial Frequency: Pixel-wise Temporal Frequency-based Deepfake Video Detection](https://mlsanity.com/paper/2507.02398/) (2025)
- [TalkingHeadBench: A Multi-Modal Benchmark & Analysis of Talking-Head DeepFake Detection](https://mlsanity.com/paper/2505.24866/) (2025)
- [Unlocking the Hidden Potential of CLIP in Generalizable Deepfake Detection](https://mlsanity.com/paper/2503.19683/) (2025)
- [Vulnerability-Aware Spatio-Temporal Learning for Generalizable Deepfake Video Detection](https://mlsanity.com/paper/2501.01184/) (2025)
- [DomainForensics: Exposing Face Forgery across Domains via Bi-directional Adaptation](https://mlsanity.com/paper/2312.10680/) (2023)
- [Exploiting Modality-Specific Features For Multi-Modal Manipulation Detection And Grounding](https://mlsanity.com/paper/2309.12657/) (2023)
- [Detecting Audio-Visual Deepfakes with Fine-Grained Inconsistencies](https://mlsanity.com/paper/2408.06753/) (2024)
- [Beyond Face Swapping: A Diffusion-Based Digital Human Benchmark for Multimodal Deepfake Detection](https://mlsanity.com/paper/2505.16512/) (2025)
- [SpeechForensics: Audio-Visual Speech Representation Learning for Face Forgery Detection](https://mlsanity.com/paper/2508.09913/) (2025)
- [DeepShield: Fortifying Deepfake Video Detection with Local and Global Forgery Analysis](https://mlsanity.com/paper/2510.25237/) (2025)

## Abstract (arXiv)

Existing face forgery detection models try to discriminate fake images by detecting only spatial artifacts (e.g., generative artifacts, blending) or mainly temporal artifacts (e.g., flickering, discontinuity). They may experience significant performance degradation when facing out-domain artifacts. In this paper, we propose to capture both spatial and temporal artifacts in one model for face forgery detection. A simple idea is to leverage a spatiotemporal model (3D ConvNet). However, we find that it may easily rely on one type of artifact and ignore the other. To address this issue, we present a novel training strategy called AltFreezing for more general face forgery detection. The AltFreezing aims to encourage the model to detect both spatial and temporal artifacts. It divides the weights of a spatiotemporal network into two groups: spatial-related and temporal-related. Then the two groups of weights are alternately frozen during the training process so that the model can learn spatial and temporal features to distinguish real or fake videos. Furthermore, we introduce various video-level data augmentation methods to improve the generalization capability of the forgery detection model. Extensive experiments show that our framework outperforms existing methods in terms of generalization to unseen manipulations and datasets. Code is available at https: //github.com/ZhendongWang6/AltFreezing.