# VoiceWukong: Benchmarking Deepfake Voice Detection

arXiv: https://arxiv.org/abs/2409.06348 · HTML: https://mlsanity.com/paper/2409.06348/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 54 of its 82 references that are in the map.
- First posted on arXiv 10 Sep 2024.

## Builds on (most cited first, 30 of 54)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [DARTS: Differentiable Architecture Search](https://mlsanity.com/paper/1806.09055/) (2018)
- [XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale](https://mlsanity.com/paper/2111.09296/) (2021)
- [Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech](https://mlsanity.com/paper/2106.06103/) (2021)
- [Qwen2-Audio Technical Report](https://mlsanity.com/paper/2407.10759/) (2024)
- [Res2Net: A New Multi-scale Backbone Architecture](https://mlsanity.com/paper/1904.01169/) (2019)
- [Large Language Models for Software Engineering: A Systematic Literature Review](https://mlsanity.com/paper/2308.10620/) (2023)
- [AISHELL-1: An Open-Source Mandarin Speech Corpus and A Speech Recognition Baseline](https://mlsanity.com/paper/1709.05522/) (2017)
- [AASIST: Audio Anti-Spoofing using Integrated Spectro-Temporal Graph Attention Networks](https://mlsanity.com/paper/2110.01200/) (2021)
- [ASVspoof 2019: Future Horizons in Spoofed and Fake Audio Detection](https://mlsanity.com/paper/1904.05441/) (2019)
- [Speak Foreign Languages with Your Own Voice: Cross-Lingual Neural Codec Language Modeling](https://mlsanity.com/paper/2303.03926/) (2023)
- [End-to-end anti-spoofing with RawNet2](https://mlsanity.com/paper/2011.01108/) (2020)
- [ASVspoof 2021: accelerating progress in spoofed and deepfake speech detection](https://mlsanity.com/paper/2109.00537/) (2021)
- [Does Audio Deepfake Detection Generalize?](https://mlsanity.com/paper/2203.16263/) (2022)
- [Automatic speaker verification spoofing and deepfake detection using wav2vec 2.0 and data augmentation](https://mlsanity.com/paper/2202.12233/) (2022)
- [AISHELL-3: A Multi-speaker Mandarin TTS Corpus and the Baselines](https://mlsanity.com/paper/2010.11567/) (2020)
- [ADD 2022: the First Audio Deep Synthesis Detection Challenge](https://mlsanity.com/paper/2202.08433/) (2022)
- [WaveFake: A Data Set to Facilitate Audio Deepfake Detection](https://mlsanity.com/paper/2111.02813/) (2021)
- [End-to-End Spectro-Temporal Graph Attention Networks for Speaker Verification Anti-Spoofing and Speech Deepfake Detection](https://mlsanity.com/paper/2107.12710/) (2021)
- [RawBoost: A Raw Data Boosting and Augmentation Method applied to Automatic Speaker Verification Anti-Spoofing](https://mlsanity.com/paper/2111.04433/) (2021)
- [ADD 2023: the Second Audio Deepfake Detection Challenge](https://mlsanity.com/paper/2305.13774/) (2023)
- [Better speech synthesis through scaling](https://mlsanity.com/paper/2305.07243/) (2023)
- [One-class Learning Towards Synthetic Voice Spoofing Detection](https://mlsanity.com/paper/2010.13995/) (2020)
- [AISHELL-4: An Open Source Dataset for Speech Enhancement, Separation, Recognition and Speaker Diarization in Conference Scenario](https://mlsanity.com/paper/2104.03603/) (2021)
- [JSUT corpus: free large-scale Japanese speech corpus for end-to-end speech synthesis](https://mlsanity.com/paper/1711.00354/) (2017)
- [StarGANv2-VC: A Diverse, Unsupervised, Non-parallel Framework for Natural-Sounding Voice Conversion](https://mlsanity.com/paper/2107.10394/) (2021)
- [A Comparative Study on Recent Neural Spoofing Countermeasures for Synthetic Speech Detection](https://mlsanity.com/paper/2103.11326/) (2021)
- [Audio Deepfake Detection: A Survey](https://mlsanity.com/paper/2308.14970/) (2023)
- [A Continual Deepfake Detection Benchmark: Dataset, Methods, and Essentials](https://mlsanity.com/paper/2205.05467/) (2022)

## Built on by (most cited first, 5 of 5)

- [Speech DF Arena: A Leaderboard for Speech DeepFake Detection Models](https://mlsanity.com/paper/2509.02859/) (2025)
- [Survey on AI-Generated Media Detection: From Non-MLLM to MLLM](https://mlsanity.com/paper/2502.05240/) (2025)
- [Ethical and Technical Limits of Deepfake Speech Datasets](https://mlsanity.com/paper/2606.10911/) (2026)
- [Can Current Detectors Catch Face-to-Voice Deepfake Attacks?](https://mlsanity.com/paper/2510.21004/) (2025)
- [VoxENES 2026: Benchmarking Generalization of Speech Spoofing Detectors Against LLM-Era TTS and Voice Conversion](https://mlsanity.com/paper/2607.11706/) (2026)

## Abstract (arXiv)

With the rapid advancement of technologies like text-to-speech (TTS) and voice conversion (VC), detecting deepfake voices has become increasingly crucial. However, both academia and industry lack a comprehensive and intuitive benchmark for evaluating detectors. Existing datasets are limited in language diversity and lack many manipulations encountered in real-world production environments. To fill this gap, we propose VoiceWukong, a benchmark designed to evaluate the performance of deepfake voice detectors. To build the dataset, we first collected deepfake voices generated by 19 advanced and widely recognized commercial tools and 15 open-source tools. We then created 38 data variants covering six types of manipulations, constructing the evaluation dataset for deepfake voice detection. VoiceWukong thus includes 265,200 English and 148,200 Chinese deepfake voice samples. Using VoiceWukong, we evaluated 12 state-of-the-art detectors. AASIST2 achieved the best equal error rate (EER) of 13.50%, while all others exceeded 20%. Our findings reveal that these detectors face significant challenges in real-world applications, with dramatically declining performance. In addition, we conducted a user study with more than 300 participants. The results are compared with the performance of the 12 detectors and a multimodel large language model (MLLM), i.e., Qwen2-Audio, where different detectors and humans exhibit varying identification capabilities for deepfake voices at different deception levels, while the LALM demonstrates no detection ability at all. Furthermore, we provide a leaderboard for deepfake voice detection, publicly available at {https://voicewukong.github.io}.