# TalkingHeadBench: A Multi-Modal Benchmark & Analysis of Talking-Head DeepFake Detection

arXiv: https://arxiv.org/abs/2505.24866 · HTML: https://mlsanity.com/paper/2505.24866/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 58 references that are in the map.
- First posted on arXiv 30 May 2025.

## Builds on (most cited first, 30 of 44)

- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [FaceForensics++: Learning to Detect Manipulated Facial Images](https://mlsanity.com/paper/1901.08971/) (2019)
- [Detecting Deepfakes with Self-Blended Images](https://mlsanity.com/paper/2204.08376/) (2022)
- [FSGAN: Subject Agnostic Face Swapping and Reenactment](https://mlsanity.com/paper/1908.05932/) (2019)
- [MAGI-1: Autoregressive Video Generation at Scale](https://mlsanity.com/paper/2505.13211/) (2025)
- [CelebV-HQ: A Large-Scale Video Facial Attributes Dataset](https://mlsanity.com/paper/2207.12393/) (2022)
- [SimSwap: An Efficient Framework For High Fidelity Face Swapping](https://mlsanity.com/paper/2106.06340/) (2021)
- [LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Control](https://mlsanity.com/paper/2407.03168/) (2024)
- [Hallo: Hierarchical Audio-Driven Visual Synthesis for Portrait Image Animation](https://mlsanity.com/paper/2406.08801/) (2024)
- [AniPortrait: Audio-Driven Synthesis of Photorealistic Portrait Animation](https://mlsanity.com/paper/2403.17694/) (2024)
- [FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Dataset](https://mlsanity.com/paper/2108.05080/) (2021)
- [DeepfakeBench: A Comprehensive Benchmark of Deepfake Detection](https://mlsanity.com/paper/2307.01426/) (2023)
- [Hierarchical Fine-Grained Image Forgery Detection and Localization](https://mlsanity.com/paper/2303.17111/) (2023)
- [DeepFaceLab: Integrated, flexible and extensible face-swapping framework](https://mlsanity.com/paper/2005.05535/) (2020)
- [DF40: Toward Next-Generation Deepfake Detection](https://mlsanity.com/paper/2406.13495/) (2024)
- [Implicit Identity Leakage: The Stumbling Block to Improving Deepfake Detection Generalization](https://mlsanity.com/paper/2210.14457/) (2022)
- [Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformer](https://mlsanity.com/paper/2412.00733/) (2024)
- [TALL: Thumbnail Layout for Deepfake Video Detection](https://mlsanity.com/paper/2307.07494/) (2023)
- [Hallo2: Long-Duration and High-Resolution Audio-Driven Portrait Image Animation](https://mlsanity.com/paper/2410.07718/) (2024)
- [X-Portrait: Expressive Portrait Animation with Hierarchical Motion Attention](https://mlsanity.com/paper/2403.15931/) (2024)
- [LAA-Net: Localized Artifact Attention Network for Quality-Agnostic and Generalizable Deepfake Detection](https://mlsanity.com/paper/2401.13856/) (2024)
- [AV-Deepfake1M: A Large-Scale LLM-Driven Audio-Visual Deepfake Dataset](https://mlsanity.com/paper/2311.15308/) (2023)
- [AltFreezing for More General Video Face Forgery Detection](https://mlsanity.com/paper/2307.08317/) (2023)
- [EMOPortraits: Emotion-enhanced Multimodal One-shot Head Avatars](https://mlsanity.com/paper/2404.19110/) (2024)
- [Implicit Identity Representation Conditioned Memory Compensation Network for Talking Head video Generation](https://mlsanity.com/paper/2307.09906/) (2023)
- [Exploiting Style Latent Flows for Generalizing Deepfake Video Detection](https://mlsanity.com/paper/2403.06592/) (2024)
- [Deepfake-Eval-2024: A Multi-Modal In-the-Wild Benchmark of Deepfakes Circulated in 2024](https://mlsanity.com/paper/2503.02857/) (2025)
- [FaceDancer: Pose- and Occlusion-Aware High Fidelity Face Swapping](https://mlsanity.com/paper/2210.10473/) (2022)
- [Glitch in the Matrix: A Large Scale Benchmark for Content Driven Audio-Visual Forgery Detection and Localization](https://mlsanity.com/paper/2305.01979/) (2023)

## Built on by (most cited first, 11 of 11)

- [Celeb-DF++: A Large-scale Challenging Video DeepFake Benchmark for Generalizable Forensics](https://mlsanity.com/paper/2507.18015/) (2025)
- [Veritas: Generalizable Deepfake Detection via Pattern-Aware Reasoning](https://mlsanity.com/paper/2508.21048/) (2025)
- [VideoVeritas: AI-Generated Video Detection via Perception Pretext Reinforcement Learning](https://mlsanity.com/paper/2602.08828/) (2026)
- [KLASSify to Verify: Audio-Visual Deepfake Detection Using SSL-based Audio and Handcrafted Visual Features](https://mlsanity.com/paper/2508.07337/) (2025)
- [Over++: Generative Video Compositing for Layer Interaction Effects](https://mlsanity.com/paper/2512.19661/) (2025)
- [SPLIT: Training-Free AI-Generated and Partially Edited Video Detection via Spatial Patch-Level Incoherence and Temporal Roughness](https://mlsanity.com/paper/2607.02886/) (2026)
- [Enhancing Self-Supervised Talking Head Forgery Detection via a Training-Free Dual-System Framework](https://mlsanity.com/paper/2605.03390/) (2026)
- [From Talking to Singing: A New Challenge for Audio-Visual Deepfake Detection](https://mlsanity.com/paper/2605.27944/) (2026)
- [DF26: We Cannot Tell Fake From Real Anymore](https://mlsanity.com/paper/2609.07369/) (2026)
- [Deepfakes and Synthetic Media: Generation, Detection, and Governance](https://mlsanity.com/paper/2609.25017/) (2026)
- [Ariadne's Thread of LipSync: Unraveling Forgeries via Inconsistency between Lip Motions and Head Poses](https://mlsanity.com/paper/2610.08417/) (2026)

## Abstract (arXiv)

The rapid advancement of talking-head deepfake generation fueled by advanced generative models has elevated the realism of synthetic videos to a level that poses substantial risks in domains such as media, politics, and finance. However, current benchmarks for deepfake talking-head detection fail to reflect this progress, relying on outdated generators and offering limited insight into model robustness and generalization. We introduce TalkingHeadBench, a comprehensive multi-model multi-generator benchmark and curated dataset designed to evaluate the performance of state-of-the-art detectors on the most advanced generators. Our dataset includes deepfakes synthesized by leading academic and commercial models and features carefully constructed protocols to assess generalization under distribution shifts in identity and generator characteristics. We benchmark a diverse set of existing detection methods, including CNNs, vision transformers, and temporal models, and analyze their robustness and generalization capabilities. In addition, we provide error analysis using Grad-CAM visualizations to expose common failure modes and detector biases. TalkingHeadBench is hosted on https://huggingface.co/datasets/luchaoqi/TalkingHeadBench with open access to all data splits and protocols. Our benchmark aims to accelerate research towards more robust and generalizable detection models in the face of rapidly evolving generative techniques.