# Harder or Different? Understanding Generalization of Audio Deepfake Detection

arXiv: https://arxiv.org/abs/2406.03512 · HTML: https://mlsanity.com/paper/2406.03512/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 38 references that are in the map.
- First posted on arXiv 5 Jun 2024.

## Builds on (most cited first, 27 of 27)

- [Shortcut Learning in Deep Neural Networks](https://mlsanity.com/paper/2004.07780/) (2020)
- [Do ImageNet Classifiers Generalize to ImageNet?](https://mlsanity.com/paper/1902.10811/) (2019)
- [WaveGrad: Estimating Gradients for Waveform Generation](https://mlsanity.com/paper/2009.00713/) (2020)
- [Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech](https://mlsanity.com/paper/2106.06103/) (2021)
- [FakeAVCeleb: A Novel Audio-Video Multimodal Deepfake Dataset](https://mlsanity.com/paper/2108.05080/) (2021)
- [AASIST: Audio Anti-Spoofing using Integrated Spectro-Temporal Graph Attention Networks](https://mlsanity.com/paper/2110.01200/) (2021)
- [ASVspoof 2019: Future Horizons in Spoofed and Fake Audio Detection](https://mlsanity.com/paper/1904.05441/) (2019)
- [End-to-end anti-spoofing with RawNet2](https://mlsanity.com/paper/2011.01108/) (2020)
- [ASVspoof 2021: accelerating progress in spoofed and deepfake speech detection](https://mlsanity.com/paper/2109.00537/) (2021)
- [Does Audio Deepfake Detection Generalize?](https://mlsanity.com/paper/2203.16263/) (2022)
- [ASVspoof 2021: Towards Spoofed and Deepfake Speech Detection in the Wild](https://mlsanity.com/paper/2210.02437/) (2022)
- [Automatic speaker verification spoofing and deepfake detection using wav2vec 2.0 and data augmentation](https://mlsanity.com/paper/2202.12233/) (2022)
- [Voice Conversion Challenge 2020: Intra-lingual semi-parallel and cross-lingual voice conversion](https://mlsanity.com/paper/2008.12527/) (2020)
- [End-to-End Spectro-Temporal Graph Attention Networks for Speaker Verification Anti-Spoofing and Speech Deepfake Detection](https://mlsanity.com/paper/2107.12710/) (2021)
- [The Voice Conversion Challenge 2018: Promoting Development of Parallel and Nonparallel Methods](https://mlsanity.com/paper/1804.04262/) (2018)
- [Better speech synthesis through scaling](https://mlsanity.com/paper/2305.07243/) (2023)
- [A Comparative Study on Recent Neural Spoofing Countermeasures for Synthetic Speech Detection](https://mlsanity.com/paper/2103.11326/) (2021)
- [MLAAD: The Multi-Language Audio Anti-Spoofing Dataset](https://mlsanity.com/paper/2401.09512/) (2024)
- [The PartialSpoof Database and Countermeasures for the Detection of Short Fake Speech Segments Embedded in an Utterance](https://mlsanity.com/paper/2204.05177/) (2022)
- [Half-Truth: A Partially Fake Audio Detection Dataset](https://mlsanity.com/paper/2104.03617/) (2021)
- [Speech is Silver, Silence is Golden: What do ASVspoof-trained Models Really Learn?](https://mlsanity.com/paper/2106.12914/) (2021)
- [Improved DeepFake Detection Using Whisper Features](https://mlsanity.com/paper/2306.01428/) (2023)
- [Raw Differentiable Architecture Search for Speech Deepfake and Spoofing Detection](https://mlsanity.com/paper/2107.12212/) (2021)
- [SingFake: Singing Voice Deepfake Detection](https://mlsanity.com/paper/2309.07525/) (2023)
- [CFAD: A Chinese Dataset for Fake Audio Detection](https://mlsanity.com/paper/2207.12308/) (2022)
- [Single and Multi-Speaker Cloned Voice Detection: From Perceptual to Learned Features](https://mlsanity.com/paper/2307.07683/) (2023)
- [Complex-valued neural networks for voice anti-spoofing](https://mlsanity.com/paper/2308.11800/) (2023)

## Built on by (most cited first, 9 of 9)

- [Do Compact SSL Backbones Matter for Audio Deepfake Detection? A Controlled Study with RAPTOR](https://mlsanity.com/paper/2603.06164/) (2026)
- [Mixture of Low-Rank Adapter Experts in Generalizable Audio Deepfake Detection](https://mlsanity.com/paper/2509.13878/) (2025)
- [CoReLoop: Parameter-Efficient Controlled Recurrent Refinement for Audio Deepfake Detection](https://mlsanity.com/paper/2609.19818/) (2026)
- [Investigating self-supervised representations for audio-visual deepfake detection](https://mlsanity.com/paper/2511.17181/) (2025)
- [The Watermark Shortcut: How Provenance Marking Sabotages Audio Deepfake Detection](https://mlsanity.com/paper/2606.23335/) (2026)
- [Teffic-Audio: Tell Fact from Fiction](https://mlsanity.com/paper/2607.28351/) (2026)
- [Towards Reliable Audio Deepfake Attribution and Model Recognition: A Multi-Level Autoencoder-Based Framework](https://mlsanity.com/paper/2508.02521/) (2025)
- [HuLA: Prosody-Aware Anti-Spoofing with Multi-Task Learning for Expressive and Emotional Synthetic Speech](https://mlsanity.com/paper/2509.21676/) (2025)
- [Detecting Audio Deepfakes on the Edge:Lightweight SSL-Based Detection in a Browser Plugin](https://mlsanity.com/paper/2606.30780/) (2026)

## Abstract (arXiv)

Recent research has highlighted a key issue in speech deepfake detection: models trained on one set of deepfakes perform poorly on others. The question arises: is this due to the continuously improving quality of Text-to-Speech (TTS) models, i.e., are newer DeepFakes just 'harder' to detect? Or, is it because deepfakes generated with one model are fundamentally different to those generated using another model? We answer this question by decomposing the performance gap between in-domain and out-of-domain test data into 'hardness' and 'difference' components. Experiments performed using ASVspoof databases indicate that the hardness component is practically negligible, with the performance gap being attributed primarily to the difference component. This has direct implications for real-world deepfake detection, highlighting that merely increasing model capacity, the currently-dominant research trend, may not effectively address the generalization challenge.