# Improving Generalization in Deepfake Detection with Face Foundation Models and Metric Learning

arXiv: https://arxiv.org/abs/2508.19730 · HTML: https://mlsanity.com/paper/2508.19730/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 44 of its 64 references that are in the map.
- First posted on arXiv 27 Aug 2025.

## Builds on (most cited first, 30 of 44)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [FaceNet: A Unified Embedding for Face Recognition and Clustering](https://mlsanity.com/paper/1503.03832/) (2015)
- [VGGFace2: A dataset for recognising faces across pose and age](https://mlsanity.com/paper/1710.08092/) (2017)
- [In Defense of the Triplet Loss for Person Re-Identification](https://mlsanity.com/paper/1703.07737/) (2017)
- [First Order Motion Model for Image Animation](https://mlsanity.com/paper/2003.00196/) (2020)
- [A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild](https://mlsanity.com/paper/2008.10010/) (2020)
- [Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensics](https://mlsanity.com/paper/1909.12962/) (2019)
- [Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Clues](https://mlsanity.com/paper/2007.09355/) (2020)
- [Detecting Deepfakes with Self-Blended Images](https://mlsanity.com/paper/2204.08376/) (2022)
- [FSGAN: Subject Agnostic Face Swapping and Reenactment](https://mlsanity.com/paper/1908.05932/) (2019)
- [Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detection](https://mlsanity.com/paper/2012.07657/) (2020)
- [DeeperForensics-1.0: A Large-Scale Dataset for Real-World Face Forgery Detection](https://mlsanity.com/paper/2001.03024/) (2020)
- [DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detection](https://mlsanity.com/paper/2001.00179/) (2020)
- [FaceShifter: Towards High Fidelity And Occlusion Aware Face Swapping](https://mlsanity.com/paper/1912.13457/) (2019)
- [WildDeepfake: A Challenging Real-World Dataset for Deepfake Detection](https://mlsanity.com/paper/2101.01456/) (2021)
- [UCF: Uncovering Common Features for Generalizable Deepfake Detection](https://mlsanity.com/paper/2304.13949/) (2023)
- [Latent Image Animator: Learning to Animate Images via Latent Space Navigation](https://mlsanity.com/paper/2203.09043/) (2022)
- [RetinaFace: Single-stage Dense Face Localisation in the Wild](https://mlsanity.com/paper/1905.00641/) (2019)
- [DeepFaceLab: Integrated, flexible and extensible face-swapping framework](https://mlsanity.com/paper/2005.05535/) (2020)
- [Leveraging Real Talking Faces via Self-Supervision for Robust Forgery Detection](https://mlsanity.com/paper/2201.07131/) (2022)
- [DF40: Toward Next-Generation Deepfake Detection](https://mlsanity.com/paper/2406.13495/) (2024)
- [ForgeryNet: A Versatile Benchmark for Comprehensive Forgery Analysis](https://mlsanity.com/paper/2103.05630/) (2021)
- [FaceForensics: A Large-scale Video Dataset for Forgery Detection in Human Faces](https://mlsanity.com/paper/1803.09179/) (2018)
- [Seeing What You Said: Talking Face Generation Guided by a Lip Reading Expert](https://mlsanity.com/paper/2303.17480/) (2023)
- [LAA-Net: Localized Artifact Attention Network for Quality-Agnostic and Generalizable Deepfake Detection](https://mlsanity.com/paper/2401.13856/) (2024)
- [Self-Supervised Video Forensics by Audio-Visual Anomaly Detection](https://mlsanity.com/paper/2301.01767/) (2023)
- [Improved Embeddings with Easy Positive Triplet Mining](https://mlsanity.com/paper/1904.04370/) (2019)
- [AVFF: Audio-Visual Feature Fusion for Video Deepfake Detection](https://mlsanity.com/paper/2406.02951/) (2024)

## Abstract (arXiv)

The increasing realism and accessibility of deepfakes have raised critical concerns about media authenticity and information integrity. Despite recent advances, deepfake detection models often struggle to generalize beyond their training distributions, particularly when applied to media content found in the wild. In this work, we present a robust video deepfake detection framework with strong generalization that takes advantage of the rich facial representations learned by face foundation models. Our method is built on top of FSFM, a self-supervised model trained on real face data, and is further fine-tuned using an ensemble of deepfake datasets spanning both face-swapping and face-reenactment manipulations. To enhance discriminative power, we incorporate triplet loss variants during training, guiding the model to produce more separable embeddings between real and fake samples. Additionally, we explore attribution-based supervision schemes, where deepfakes are categorized by manipulation type or source dataset, to assess their impact on generalization. Extensive experiments across diverse evaluation benchmarks demonstrate the effectiveness of our approach, especially in challenging real-world scenarios.