# Exploring Unbiased Deepfake Detection via Token-Level Shuffling and Mixing

arXiv: https://arxiv.org/abs/2501.04376 · HTML: https://mlsanity.com/paper/2501.04376/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 49 references that are in the map.
- First posted on arXiv 8 Jan 2025.

## Builds on (most cited first, 30 of 38)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [FaceForensics++: Learning to Detect Manipulated Facial Images](https://mlsanity.com/paper/1901.08971/) (2019)
- [Grad-CAM++: Improved Visual Explanations for Deep Convolutional Networks](https://mlsanity.com/paper/1710.11063/) (2017)
- [Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Clues](https://mlsanity.com/paper/2007.09355/) (2020)
- [Face X-ray for More General Face Forgery Detection](https://mlsanity.com/paper/1912.13458/) (2019)
- [Detecting Deepfakes with Self-Blended Images](https://mlsanity.com/paper/2204.08376/) (2022)
- [Exposing DeepFake Videos By Detecting Face Warping Artifacts](https://mlsanity.com/paper/1811.00656/) (2018)
- [Exposing Deep Fakes Using Inconsistent Head Poses](https://mlsanity.com/paper/1811.00661/) (2018)
- [Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detection](https://mlsanity.com/paper/2012.07657/) (2020)
- [What makes fake images detectable? Understanding properties that generalize](https://mlsanity.com/paper/2008.10588/) (2020)
- [Generalizing Face Forgery Detection with High-frequency Features](https://mlsanity.com/paper/2103.12376/) (2021)
- [On the Detection of Digital Face Manipulation](https://mlsanity.com/paper/1910.01717/) (2019)
- [DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detection](https://mlsanity.com/paper/2001.03024/) (2020)
- [FaceShifter: Towards High Fidelity And Occlusion Aware Face Swapping](https://mlsanity.com/paper/1912.13457/) (2019)
- [The Deepfake Detection Challenge (DFDC) Preview Dataset](https://mlsanity.com/paper/1910.08854/) (2019)
- [Representation Learning via Invariant Causal Mechanisms](https://mlsanity.com/paper/2010.07922/) (2020)
- [Learning Self-Consistency for Deepfake Detection](https://mlsanity.com/paper/2012.09311/) (2020)
- [Self-supervised Learning of Adversarial Example: Towards Good Generalizations for Deepfake Detection](https://mlsanity.com/paper/2203.12208/) (2022)
- [DeepfakeBench: A Comprehensive Benchmark of Deepfake Detection](https://mlsanity.com/paper/2307.01426/) (2023)
- [Exploring Temporal Coherence for More General Video Face Forgery Detection](https://mlsanity.com/paper/2108.06693/) (2021)
- [Frequency-aware Discriminative Feature Learning Supervised by Single-Center Loss for Face Forgery Detection](https://mlsanity.com/paper/2103.09096/) (2021)
- [DF40: Toward Next-Generation Deepfake Detection](https://mlsanity.com/paper/2406.13495/) (2024)
- [Transcending Forgery Specificity with Latent Space Augmentation for Generalizable Deepfake Detection](https://mlsanity.com/paper/2311.11278/) (2023)
- [Implicit Identity Leakage: The Stumbling Block to Improving Deepfake Detection Generalization](https://mlsanity.com/paper/2210.14457/) (2022)
- [Protecting Celebrities from DeepFake with Identity Consistency Transformer](https://mlsanity.com/paper/2203.01318/) (2022)

## Built on by (most cited first, 27 of 27)

- [Unlocking the Capabilities of Large Vision-Language Models for Generalizable and Explainable Deepfake Detection](https://mlsanity.com/paper/2503.14853/) (2025)
- [Deepfake Detection that Generalizes Across Benchmarks](https://mlsanity.com/paper/2508.06248/) (2025)
- [VideoVeritas: AI-Generated Video Detection via Perception Pretext Reinforcement Learning](https://mlsanity.com/paper/2602.08828/) (2026)
- [Unlocking the Hidden Potential of CLIP in Generalizable Deepfake Detection](https://mlsanity.com/paper/2503.19683/) (2025)
- [Towards Generalizable AI-Generated Image Detection via Image-Adaptive Prompt Learning](https://mlsanity.com/paper/2508.01603/) (2025)
- [NS-Net: Decoupling CLIP Semantic Information through NULL-Space for Generalizable AI-Generated Image Detection](https://mlsanity.com/paper/2508.01248/) (2025)
- [DGS-Net: Distillation-Guided Gradient Surgery for CLIP Fine-Tuning in AI-Generated Image Detection](https://mlsanity.com/paper/2511.13108/) (2025)
- [Beyond Semantic Features: Pixel-level Mapping for Generalized AI-Generated Image Detection](https://mlsanity.com/paper/2512.17350/) (2025)
- [WMamba: Wavelet-based Mamba for Face Forgery Detection](https://mlsanity.com/paper/2501.09617/) (2025)
- [VRAG-DFD: Verifiable Retrieval-Augmentation for MLLM-based Deepfake Detection](https://mlsanity.com/paper/2604.13660/) (2026)
- [When Detectors Forget Forensics: Blocking Semantic Shortcuts for Generalizable AI-Generated Image Detection](https://mlsanity.com/paper/2603.09242/) (2026)
- [LOGER: Local--Global Ensemble for Robust Deepfake Detection in the Wild](https://mlsanity.com/paper/2604.03558/) (2026)
- [HydraPrompt: An Adaptive and Asymmetric Framework of Vision-Language Models for Synthetic Image Detection](https://mlsanity.com/paper/2605.26421/) (2026)
- [Uncertainty-Aware Deepfake Detection via Multi-View Structural Learning](https://mlsanity.com/paper/2607.28769/) (2026)
- [VidForensics-M1: Meta-Detection Reinforcement Learning with Verifiable Temporal Grounding for AI-Generated Video Forensics](https://mlsanity.com/paper/2608.11201/) (2026)
- [DiffusionFF: A Diffusion-based Framework for Joint Face Forgery Detection and Fine-Grained Artifact Localization](https://mlsanity.com/paper/2508.01873/) (2025)
- [Leveraging Unlabeled Data from Unknown Sources via Dual-Path Guidance for Deepfake Face Detection](https://mlsanity.com/paper/2508.09022/) (2025)
- [LAKAN: Landmark-assisted Adaptive Kolmogorov-Arnold Network for Face Forgery Detection](https://mlsanity.com/paper/2510.00634/) (2025)
- [Proto-LeakNet: Towards Signal-Leak Aware Attribution in Synthetic Human Face Imagery](https://mlsanity.com/paper/2511.04260/) (2025)
- [Low-rank Orthogonal Subspace Intervention for Generalizable Face Forgery Detection](https://mlsanity.com/paper/2601.11915/) (2026)
- [Unleashing Vision-Language Semantics for Deepfake Video Detection](https://mlsanity.com/paper/2603.24454/) (2026)
- [LAA-X: Unified Localized Artifact Attention for Quality-Agnostic and Generalizable Face Forgery Detection](https://mlsanity.com/paper/2604.04086/) (2026)
- [Generalizable Face Forgery Detection via Separable Prompt Learning](https://mlsanity.com/paper/2604.17307/) (2026)
- [Robust Deepfake Detection: Mitigating Spatial Attention Drift via Calibrated Complementary Ensembles](https://mlsanity.com/paper/2604.25889/) (2026)
- [SGF-CDNet: A Consistency-Discrepancy Graph Network over Semantic-Geometric Fused Nodes for Face Forgery Detection](https://mlsanity.com/paper/2607.03883/) (2026)
- [DF-CBM: Region-Aware Concept Bottleneck Models for Deepfake Detection](https://mlsanity.com/paper/2609.35096/) (2026)
- [Revisiting Cross-Reconstruction for Generalizable Deepfake Detection](https://mlsanity.com/paper/2610.01544/) (2026)

## Abstract (arXiv)

The generalization problem is broadly recognized as a critical challenge in detecting deepfakes. Most previous work believes that the generalization gap is caused by the differences among various forgery methods. However, our investigation reveals that the generalization issue can still occur when forgery-irrelevant factors shift. In this work, we identify two biases that detectors may also be prone to overfitting: position bias and content bias, as depicted in Fig. 1. For the position bias, we observe that detectors are prone to lazily depending on the specific positions within an image (e.g., central regions even no forgery). As for content bias, we argue that detectors may potentially and mistakenly utilize forgery-unrelated information for detection (e.g., background, and hair). To intervene these biases, we propose two branches for shuffling and mixing with tokens in the latent space of transformers. For the shuffling branch, we rearrange the tokens and corresponding position embedding for each image while maintaining the local correlation. For the mixing branch, we randomly select and mix the tokens in the latent space between two images with the same label within the mini-batch to recombine the content information. During the learning process, we align the outputs of detectors from different branches in both feature space and logit space. Contrastive losses for features and divergence losses for logits are applied to obtain unbiased feature representation and classifiers. We demonstrate and verify the effectiveness of our method through extensive experiments on widely used evaluation datasets.