# Voice-Face Homogeneity Tells Deepfake

arXiv: https://arxiv.org/abs/2203.02195 · HTML: https://mlsanity.com/paper/2203.02195/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 98 references that are in the map.
- First posted on arXiv 4 Mar 2022.

## Builds on (most cited first, 30 of 47)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [FaceForensics++: Learning to Detect Manipulated Facial Images](https://mlsanity.com/paper/1901.08971/) (2019)
- [VoxCeleb2: Deep Speaker Recognition](https://mlsanity.com/paper/1806.05622/) (2018)
- [VoxCeleb: a large-scale speaker identification dataset](https://mlsanity.com/paper/1706.08612/) (2017)
- [Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks](https://mlsanity.com/paper/1703.10593/) (2017)
- [A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wild](https://mlsanity.com/paper/2008.10010/) (2020)
- [Celeb-DF: A Large-scale Challenging Dataset for DeepFake Forensics](https://mlsanity.com/paper/1909.12962/) (2019)
- [Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Clues](https://mlsanity.com/paper/2007.09355/) (2020)
- [Face X-ray for More General Face Forgery Detection](https://mlsanity.com/paper/1912.13458/) (2019)
- [Deep metric learning using Triplet network](https://mlsanity.com/paper/1412.6622/) (2014)
- [MesoNet: a Compact Facial Video Forgery Detection Network](https://mlsanity.com/paper/1809.00888/) (2018)
- [Multi-attentional Deepfake Detection](https://mlsanity.com/paper/2103.02406/) (2021)
- [MagFace: A Universal Representation for Face Recognition and Quality Assessment](https://mlsanity.com/paper/2103.06627/) (2021)
- [FSGAN: Subject Agnostic Face Swapping and Reenactment](https://mlsanity.com/paper/1908.05932/) (2019)
- [Exposing DeepFake Videos By Detecting Face Warping Artifacts](https://mlsanity.com/paper/1811.00656/) (2018)
- [Exposing Deep Fakes Using Inconsistent Head Poses](https://mlsanity.com/paper/1811.00661/) (2018)
- [Lips Don't Lie: A Generalisable and Robust Approach to Face Forgery Detection](https://mlsanity.com/paper/2012.07657/) (2020)
- [Deep Video Portraits](https://mlsanity.com/paper/1805.11714/) (2018)
- [Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domain](https://mlsanity.com/paper/2103.01856/) (2021)
- [DeepFakes: a New Threat to Face Recognition? Assessment and Detection](https://mlsanity.com/paper/1812.08685/) (2018)
- [FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Dataset](https://mlsanity.com/paper/2108.05080/) (2021)
- [Capsule-Forensics: Using Capsule Networks to Detect Forged Images and Videos](https://mlsanity.com/paper/1810.11215/) (2018)
- [Two-branch Recurrent Network for Isolating Deepfakes in Videos](https://mlsanity.com/paper/2008.03412/) (2020)

## Built on by (most cited first, 30 of 44)

- [AVFF: Audio-Visual Feature Fusion for Video Deepfake Detection](https://mlsanity.com/paper/2406.02951/) (2024)
- [Deepfake Detection: A Comprehensive Survey from the Reliability Perspective](https://mlsanity.com/paper/2211.10881/) (2022)
- [Diffusion Facial Forgery Detection](https://mlsanity.com/paper/2401.15859/) (2024)
- [Parents and Children: Distinguishing Multimodal DeepFakes from Natural Images](https://mlsanity.com/paper/2304.00500/) (2023)
- [Deepfake Video Detection with Spatiotemporal Dropout Transformer](https://mlsanity.com/paper/2207.06612/) (2022)
- [DiMoDif: Discourse Modality-information Differentiation for Audio-visual Deepfake Detection and Localization](https://mlsanity.com/paper/2411.10193/) (2024)
- [MIS-AVoiDD: Modality Invariant and Specific Representation for Audio-Visual Deepfake Detection](https://mlsanity.com/paper/2310.02234/) (2023)
- [Integrating Audio-Visual Features for Multimodal Deepfake Detection](https://mlsanity.com/paper/2310.03827/) (2023)
- [AVT2-DWF: Improving Deepfake Detection with Audio-Visual Fusion and Dynamic Weighting Strategies](https://mlsanity.com/paper/2403.14974/) (2024)
- [FakeOut: Leveraging Out-of-domain Self-supervision for Multi-modal Video Deepfake Detection](https://mlsanity.com/paper/2212.00773/) (2022)
- [Real Face Foundation Representation Learning for Generalized Deepfake Detection](https://mlsanity.com/paper/2303.08439/) (2023)
- [Detecting Audio-Visual Deepfakes with Fine-Grained Inconsistencies](https://mlsanity.com/paper/2408.06753/) (2024)
- [Open-Set Deepfake Detection: A Parameter-Efficient Adaptation Method with Forgery Style Mixture](https://mlsanity.com/paper/2408.12791/) (2024)
- [Fair Deepfake Detectors Can Generalize](https://mlsanity.com/paper/2507.02645/) (2025)
- [SpeechForensics: Audio-Visual Speech Representation Learning for Face Forgery Detection](https://mlsanity.com/paper/2508.09913/) (2025)
- [Evolving from Single-modal to Multi-modal Facial Deepfake Detection: Progress and Challenges](https://mlsanity.com/paper/2406.06965/) (2024)
- [Contextual Cross-Modal Attention for Audio-Visual Deepfake Detection and Localization](https://mlsanity.com/paper/2408.01532/) (2024)
- [Learning Real Facial Concepts for Independent Deepfake Detection](https://mlsanity.com/paper/2505.04460/) (2025)
- [CAD: A General Multimodal Framework for Video Deepfake Detection via Cross-Modal Alignment and Distillation](https://mlsanity.com/paper/2505.15233/) (2025)
- [RetouchingFFHQ: A Large-scale Dataset for Fine-grained Face Retouching Detection](https://mlsanity.com/paper/2307.10642/) (2023)
- [PIA: Deepfake Detection Using Phoneme-Temporal and Identity-Dynamic Analysis](https://mlsanity.com/paper/2510.14241/) (2025)
- [NPVForensics: Jointing Non-critical Phonemes and Visemes for Deepfake Detection](https://mlsanity.com/paper/2306.06885/) (2023)
- [Passive Deepfake Detection Across Multi-modalities: A Comprehensive Survey](https://mlsanity.com/paper/2411.17911/) (2024)
- [Audio-Visual Deepfake Detection With Local Temporal Inconsistencies](https://mlsanity.com/paper/2501.08137/) (2025)
- [Statistics-aware Audio-visual Deepfake Detector](https://mlsanity.com/paper/2407.11650/) (2024)
- [Big Brother is Watching: Proactive Deepfake Detection via Learnable Hidden Face](https://mlsanity.com/paper/2504.11309/) (2025)
- [Global Prompt Refinement with Non-Interfering Attention Masking for One-Shot Federated Learning](https://mlsanity.com/paper/2509.22700/) (2025)
- [Next-Frame Feature Prediction for Multimodal Deepfake Detection and Temporal Localization](https://mlsanity.com/paper/2511.10212/) (2025)
- [Deepfake Face Traceability with Disentangling Reversing Network](https://mlsanity.com/paper/2207.03666/) (2022)
- [Towards Generalizable Deepfake Detection by Primary Region Regularization](https://mlsanity.com/paper/2307.12534/) (2023)

## Abstract (arXiv)

Detecting forgery videos is highly desirable due to the abuse of deepfake. Existing detection approaches contribute to exploring the specific artifacts in deepfake videos and fit well on certain data. However, the growing technique on these artifacts keeps challenging the robustness of traditional deepfake detectors. As a result, the development of generalizability of these approaches has reached a blockage. To address this issue, given the empirical results that the identities behind voices and faces are often mismatched in deepfake videos, and the voices and faces have homogeneity to some extent, in this paper, we propose to perform the deepfake detection from an unexplored voice-face matching view. To this end, a voice-face matching method is devised to measure the matching degree of these two. Nevertheless, training on specific deepfake datasets makes the model overfit certain traits of deepfake algorithms. We instead, advocate a method that quickly adapts to untapped forgery, with a pre-training then fine-tuning paradigm. Specifically, we first pre-train the model on a generic audio-visual dataset, followed by the fine-tuning on downstream deepfake data. We conduct extensive experiments over three widely exploited deepfake datasets - DFDC, FakeAVCeleb, and DeepfakeTIMIT. Our method obtains significant performance gains as compared to other state-of-the-art competitors. It is also worth noting that our method already achieves competitive results when fine-tuned on limited deepfake data.