# Protecting Celebrities from DeepFake with Identity Consistency Transformer

arXiv: https://arxiv.org/abs/2203.01318 · HTML: https://mlsanity.com/paper/2203.01318/

## Key facts

- Cited by 95 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 62 references that are in the map.
- First posted on arXiv 2 Mar 2022.

## Builds on (most cited first, 30 of 44)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [FaceForensics++: Learning to Detect Manipulated Facial Images](https://mlsanity.com/paper/1901.08971/) (2019)
- [CosFace: Large Margin Cosine Loss for Deep Face Recognition](https://mlsanity.com/paper/1801.09414/) (2018)
- [CNN-generated images are surprisingly easy to spot... for now](https://mlsanity.com/paper/1912.11035/) (2019)
- [Face2Face: Real-time Face Capture and Reenactment of RGB Videos](https://mlsanity.com/paper/2007.14808/) (2020)
- [MS-Celeb-1M: A Dataset and Benchmark for Large-Scale Face Recognition](https://mlsanity.com/paper/1607.08221/) (2016)
- [Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensics](https://mlsanity.com/paper/1909.12962/) (2019)
- [CSWin Transformer: A General Vision Transformer Backbone with Cross-Shaped Windows](https://mlsanity.com/paper/2107.00652/) (2021)
- [Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Clues](https://mlsanity.com/paper/2007.09355/) (2020)
- [Face X-ray for More General Face Forgery Detection](https://mlsanity.com/paper/1912.13458/) (2019)
- [MesoNet: a Compact Facial Video Forgery Detection Network](https://mlsanity.com/paper/1809.00888/) (2018)
- [Multi-attentional Deepfake Detection](https://mlsanity.com/paper/2103.02406/) (2021)
- [FSGAN: Subject Agnostic Face Swapping and Reenactment](https://mlsanity.com/paper/1908.05932/) (2019)
- [Exposing DeepFake Videos By Detecting Face Warping Artifacts](https://mlsanity.com/paper/1811.00656/) (2018)
- [Exposing Deep Fakes Using Inconsistent Head Poses](https://mlsanity.com/paper/1811.00661/) (2018)
- [Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detection](https://mlsanity.com/paper/2012.07657/) (2020)
- [What makes fake images detectable? Understanding properties that generalize](https://mlsanity.com/paper/2008.10588/) (2020)
- [On the Detection of Digital Face Manipulation](https://mlsanity.com/paper/1910.01717/) (2019)
- [DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detection](https://mlsanity.com/paper/2001.03024/) (2020)
- [Mobile-Former: Bridging MobileNet and Transformer](https://mlsanity.com/paper/2108.05895/) (2021)
- [FaceShifter: Towards High Fidelity And Occlusion Aware Face Swapping](https://mlsanity.com/paper/1912.13457/) (2019)
- [BEVT: BERT Pretraining of Video Transformers](https://mlsanity.com/paper/2112.01529/) (2021)
- [Two-branch Recurrent Network for Isolating Deepfakes in Videos](https://mlsanity.com/paper/2008.03412/) (2020)
- [Exploring Temporal Coherence for More General Video Face Forgery Detection](https://mlsanity.com/paper/2108.06693/) (2021)
- [Learning Rich Features for Image Manipulation Detection](https://mlsanity.com/paper/1805.04953/) (2018)
- [High-Fidelity Pluralistic Image Completion with Transformers](https://mlsanity.com/paper/2103.14031/) (2021)

## Built on by (most cited first, 30 of 95)

- [Transcending Forgery Specificity with Latent Space Augmentation for Generalizable Deepfake Detection](https://mlsanity.com/paper/2311.11278/) (2023)
- [Implicit Identity Leakage: The Stumbling Block to Improving Deepfake Detection Generalization](https://mlsanity.com/paper/2210.14457/) (2022)
- [TALL: Thumbnail Layout for Deepfake Video Detection](https://mlsanity.com/paper/2307.07494/) (2023)
- [AvatarReX: Real-time Expressive Full-body Avatars](https://mlsanity.com/paper/2305.04789/) (2023)
- [AltFreezing for More General Video Face Forgery Detection](https://mlsanity.com/paper/2307.08317/) (2023)
- [Efficient Region-Aware Neural Radiance Fields for High-Fidelity Talking Portrait Synthesis](https://mlsanity.com/paper/2307.09323/) (2023)
- [AVFF: Audio-Visual Feature Fusion for Video Deepfake Detection](https://mlsanity.com/paper/2406.02951/) (2024)
- [Beyond the Prior Forgery Knowledge: Mining Critical Clues for General Face Forgery Detection](https://mlsanity.com/paper/2304.12489/) (2023)
- [Exposing the Deception: Uncovering More Forgery Clues for Deepfake Detection](https://mlsanity.com/paper/2403.01786/) (2024)
- [Towards General Visual-Linguistic Face Forgery Detection](https://mlsanity.com/paper/2307.16545/) (2023)
- [SepMark: Deep Separable Watermarking for Unified Source Tracing and Deepfake Detection](https://mlsanity.com/paper/2305.06321/) (2023)
- [Audio-Visual Person-of-Interest DeepFake Detection](https://mlsanity.com/paper/2204.03083/) (2022)
- [Improving Fairness in Deepfake Detection](https://mlsanity.com/paper/2306.16635/) (2023)
- [PTW: Pivotal Tuning Watermarking for Pre-Trained Image Generators](https://mlsanity.com/paper/2304.07361/) (2023)
- [Robustness and Generalizability of Deepfake Detection: A Study with Diffusion Models](https://mlsanity.com/paper/2309.02218/) (2023)
- [X2-DFD: A framework for eXplainable and eXtendable Deepfake Detection](https://mlsanity.com/paper/2410.06126/) (2024)
- [BusterX: MLLM-Powered AI-Generated Video Forgery Detection and Explanation](https://mlsanity.com/paper/2505.12620/) (2025)
- [Fake It till You Make It: Curricular Dynamic Forgery Augmentations towards General Deepfake Detection](https://mlsanity.com/paper/2409.14444/) (2024)
- [DiffusionFake: Enhancing Generalization in Deepfake Detection via Guided Stable Diffusion](https://mlsanity.com/paper/2410.04372/) (2024)
- [MoE-FFD: Mixture of Experts for Generalized and Parameter-Efficient Face Forgery Detection](https://mlsanity.com/paper/2404.08452/) (2024)
- [Exploring Unbiased Deepfake Detection via Token-Level Shuffling and Mixing](https://mlsanity.com/paper/2501.04376/) (2025)
- [DeepFake-Adapter: Dual-Level Adapter for DeepFake Detection](https://mlsanity.com/paper/2306.00863/) (2023)
- [Locate and Verify: A Two-Stream Network for Improved Deepfake Detection](https://mlsanity.com/paper/2309.11131/) (2023)
- [Learning Natural Consistency Representation for Face Forgery Video Detection](https://mlsanity.com/paper/2407.10550/) (2024)
- [Weakly-supervised deepfake localization in diffusion-generated images](https://mlsanity.com/paper/2311.04584/) (2023)
- [MINTIME: Multi-Identity Size-Invariant Video Deepfake Detection](https://mlsanity.com/paper/2211.10996/) (2022)
- [DFIL: Deepfake Incremental Learning by Exploiting Domain-invariant Forgery Clues](https://mlsanity.com/paper/2309.09526/) (2023)
- [Learning Spatiotemporal Inconsistency via Thumbnail Layout for Face Deepfake Detection](https://mlsanity.com/paper/2403.10261/) (2024)
- [Deepfake Video Detection with Spatiotemporal Dropout Transformer](https://mlsanity.com/paper/2207.06612/) (2022)
- [SoK: Systematization and Benchmarking of Deepfake Detectors in a Unified Framework](https://mlsanity.com/paper/2401.04364/) (2024)

## Abstract (arXiv)

In this work we propose Identity Consistency Transformer, a novel face forgery detection method that focuses on high-level semantics, specifically identity information, and detecting a suspect face by finding identity inconsistency in inner and outer face regions. The Identity Consistency Transformer incorporates a consistency loss for identity consistency determination. We show that Identity Consistency Transformer exhibits superior generalization ability not only across different datasets but also across various types of image degradation forms found in real-world applications including deepfake videos. The Identity Consistency Transformer can be easily enhanced with additional identity information when such information is available, and for this reason it is especially well-suited for detecting face forgeries involving celebrities. Code will be released at urlhttps://github.com/LightDXY/ICTDeepFake