# Video Transformer for Deepfake Detection with Incremental Learning

arXiv: https://arxiv.org/abs/2108.05307 · HTML: https://mlsanity.com/paper/2108.05307/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 56 references that are in the map.
- First posted on arXiv 11 Aug 2021.

## Builds on (most cited first, 30 of 38)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Training data-efficient image transformers & distillation through attention](https://mlsanity.com/paper/2012.12877/) (2020)
- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://mlsanity.com/paper/2101.03961/) (2021)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Tasks](https://mlsanity.com/paper/1908.02265/) (2019)
- [Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/1511.06434/) (2015)
- [Linformer: Self-Attention with Linear Complexity](https://mlsanity.com/paper/2006.04768/) (2020)
- [FaceForensics++: Learning to Detect Manipulated Facial Images](https://mlsanity.com/paper/1901.08971/) (2019)
- [Transformers in Vision: A Survey](https://mlsanity.com/paper/2101.01169/) (2021)
- [VideoBERT: A Joint Model for Video and Language Representation Learning](https://mlsanity.com/paper/1904.01766/) (2019)
- [End-to-End Incremental Learning](https://mlsanity.com/paper/1807.09536/) (2018)
- [Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensics](https://mlsanity.com/paper/1909.12962/) (2019)
- [Face X-ray for More General Face Forgery Detection](https://mlsanity.com/paper/1912.13458/) (2019)
- [MesoNet: a Compact Facial Video Forgery Detection Network](https://mlsanity.com/paper/1809.00888/) (2018)
- [Exposing DeepFake Videos By Detecting Face Warping Artifacts](https://mlsanity.com/paper/1811.00656/) (2018)
- [Video Action Transformer Network](https://mlsanity.com/paper/1812.02707/) (2018)
- [Exposing Deep Fakes Using Inconsistent Head Poses](https://mlsanity.com/paper/1811.00661/) (2018)
- [The Creation and Detection of Deepfakes: A Survey](https://mlsanity.com/paper/2004.11138/) (2020)
- [Towards Fast, Accurate and Stable 3D Dense Face Alignment](https://mlsanity.com/paper/2009.09960/) (2020)
- [DeepFakes: a New Threat to Face Recognition? Assessment and Detection](https://mlsanity.com/paper/1812.08685/) (2018)
- [Media Forensics and DeepFakes: an overview](https://mlsanity.com/paper/2001.06564/) (2020)
- [Scaling Neural Machine Translation](https://mlsanity.com/paper/1806.00187/) (2018)
- [Capsule-Forensics: Using Capsule Networks to Detect Forged Images and Videos](https://mlsanity.com/paper/1810.11215/) (2018)
- [FakeCatcher: Detection of Synthetic Portrait Videos using Biological Signals](https://mlsanity.com/paper/1901.02212/) (2019)
- [ImageBERT: Cross-modal Pre-training with Large-scale Weak-supervised Image-Text Data](https://mlsanity.com/paper/2001.07966/) (2020)

## Built on by (most cited first, 30 of 33)

- [S-Prompts Learning with Pre-trained Transformers: An Occam's Razor for Domain Incremental Learning](https://mlsanity.com/paper/2207.12819/) (2022)
- [Leveraging Real Talking Faces via Self-Supervision for Robust Forgery Detection](https://mlsanity.com/paper/2201.07131/) (2022)
- [TALL: Thumbnail Layout for Deepfake Video Detection](https://mlsanity.com/paper/2307.07494/) (2023)
- [UIA-ViT: Unsupervised Inconsistency-Aware Method based on Vision Transformer for Face Forgery Detection](https://mlsanity.com/paper/2210.12752/) (2022)
- [Audio-Visual Person-of-Interest DeepFake Detection](https://mlsanity.com/paper/2204.03083/) (2022)
- [Learning Spatiotemporal Inconsistency via Thumbnail Layout for Face Deepfake Detection](https://mlsanity.com/paper/2403.10261/) (2024)
- [Digital and Physical Face Attacks: Reviewing and One Step Further](https://mlsanity.com/paper/2209.14692/) (2022)
- [The Age of Synthetic Realities: Challenges and Opportunities](https://mlsanity.com/paper/2306.11503/) (2023)
- [Exploring Self-Supervised Vision Transformers for Deepfake Detection: A Comparative Analysis](https://mlsanity.com/paper/2405.00355/) (2024)
- [A Review of Deep Learning-based Approaches for Deepfake Content Detection](https://mlsanity.com/paper/2202.06095/) (2022)
- [GenConViT: Deepfake Video Detection Using Generative Convolutional Vision Transformer](https://mlsanity.com/paper/2307.07036/) (2023)
- [A Timely Survey on Vision Transformer for Deepfake Detection](https://mlsanity.com/paper/2405.08463/) (2024)
- [AVTENet: A Human-Cognition-Inspired Audio-Visual Transformer-Based Ensemble Network for Video Deepfake Detection](https://mlsanity.com/paper/2310.13103/) (2023)
- [The MeVer DeepFake Detection Service: Lessons Learnt from Developing and Deploying in the Wild](https://mlsanity.com/paper/2204.12816/) (2022)
- [Hybrid Transformer Network for Deepfake Detection](https://mlsanity.com/paper/2208.05820/) (2022)
- [Multimodal Graph Learning for Deepfake Detection](https://mlsanity.com/paper/2209.05419/) (2022)
- [Self-Supervised Graph Transformer for Deepfake Detection](https://mlsanity.com/paper/2307.15019/) (2023)
- [Continuous fake media detection: adapting deepfake detectors to new generative techniques](https://mlsanity.com/paper/2406.08171/) (2024)
- [DIP: Diffusion Learning of Inconsistency Pattern for General DeepFake Detection](https://mlsanity.com/paper/2410.23663/) (2024)
- [Generalized Few-Shot Continual Learning with Contrastive Mixture of Adapters](https://mlsanity.com/paper/2302.05936/) (2023)
- [A Quality-Centric Framework for Generic Deepfake Detection](https://mlsanity.com/paper/2411.05335/) (2024)
- [Vision Transformer Based Video Hashing Retrieval for Tracing the Source of Fake Videos](https://mlsanity.com/paper/2112.08117/) (2021)
- [Deepfake Detection: A Comparative Analysis](https://mlsanity.com/paper/2308.03471/) (2023)
- [WWW: Where, Which and Whatever Enhancing Interpretability in Multimodal Deepfake Detection](https://mlsanity.com/paper/2408.02954/) (2024)
- [VGMShield: Mitigating Misuse of Video Generative Models](https://mlsanity.com/paper/2402.13126/) (2024)
- [G2V2former: Graph Guided Video Vision Transformer for Face Anti-Spoofing](https://mlsanity.com/paper/2408.07675/) (2024)
- [Trident: Detecting Face Forgeries with Adversarial Triplet Learning](https://mlsanity.com/paper/2506.23189/) (2025)
- [UMCL: Unimodal-generated Multimodal Contrastive Learning for Cross-compression-rate Deepfake Detection](https://mlsanity.com/paper/2511.18983/) (2025)
- [Grab-3D: Detecting AI-Generated Videos from 3D Geometric Temporal Consistency](https://mlsanity.com/paper/2512.13665/) (2025)
- [Learning to mask: Towards generalized face forgery detection](https://mlsanity.com/paper/2212.14309/) (2022)

## Abstract (arXiv)

Face forgery by deepfake is widely spread over the internet and this raises severe societal concerns. In this paper, we propose a novel video transformer with incremental learning for detecting deepfake videos. To better align the input face images, we use a 3D face reconstruction method to generate UV texture from a single input face image. The aligned face image can also provide pose, eyes blink and mouth movement information that cannot be perceived in the UV texture image, so we use both face images and their UV texture maps to extract the image features. We present an incremental learning strategy to fine-tune the proposed model on a smaller amount of data and achieve better deepfake detection performance. The comprehensive experiments on various public deepfake datasets demonstrate that the proposed video transformer model with incremental learning achieves state-of-the-art performance in the deepfake video detection task with enhanced feature learning from the sequenced data.