Does Current Deepfake Audio Detection Model Effectively Detect ALM-based Deepfake Audio?
Not on the map. Why?
Abstract (arXiv)
Currently, Audio Language Models (ALMs) are rapidly advancing due to the developments in large language models and audio neural codecs. These ALMs have significantly lowered the barrier to creating deepfake audio, generating highly realistic and diverse types of deepfake audio, which pose severe threats to society. Consequently, effective audio deepfake detection technologies to detect ALM-based audio have become increasingly critical. This paper investigate the effectiveness of current countermeasure (CM) against ALM-based audio. Specifically, we collect 12 types of the latest ALM-based deepfake audio and utilizing the latest CMs to evaluate. Our findings reveal that the latest codec-trained CM can effectively detect ALM-based audio, achieving 0% equal error rate under most ALM test conditions, which exceeded our expectations. This indicates promising directions for future research in ALM-based deepfake audio detection.
Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 25 of its 28 references that are in the map. First posted on arXiv 20 Aug 2024.
Builds on
25 of 28 references.
References · 25
- 2023Neural Codec Language Models are Zero-Shot Text to Speech Synthesizerscited by 737
- 2022High Fidelity Neural Audio Compressioncited by 673
- 2022AudioLM: a Language Modeling Approach to Audio Generationcited by 611
- 2023Simple and Controllable Music Generationcited by 480
- 2023MusicLM: Generating Music From Textcited by 442
- 2022AudioGen: Textually Guided Audio Generationcited by 292
- 2024NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Modelscited by 222
- 2021AASIST: Audio Anti-Spoofing using Integrated Spectro-Temporal Graph Attention Networkscited by 173
- 2023Speak Foreign Languages with Your Own Voice: Cross-Lingual Neural Codec Language Modelingcited by 166
- 2023UniAudio: An Audio Foundation Model Toward Universal Audio Generationcited by 127
- 2024VALL-E 2: Neural Codec Language Models are Human Parity Zero-Shot Text to Speech Synthesizerscited by 119
- 2022ASVspoof 2021: Towards Spoofed and Deepfake Speech Detection in the Wildcited by 118
13 more
- 2022Automatic speaker verification spoofing and deepfake detection using wav2vec 2.0 and data augmentationcited by 110
- 2023VioLA: Unified Codec Language Models for Speech Recognition, Synthesis, and Translationcited by 87
- 2023LauraGPT: Listen, Attend, Understand, and Regenerate Audio with GPTcited by 77
- 2023SpeechX: Neural Codec Language Model as a Versatile Speech Transformercited by 64
- 2021ASVspoof 2019: spoofing countermeasures for the detection of synthesized, converted and replayed speechcited by 56
- 2019STC Antispoofing Systems for the ASVspoof2019 Challengecited by 51
- 2024ELLA-V: Stable Neural Codec Language Modeling with Alignment-guided Sequence Reorderingcited by 50
- 2024The Codecfake Dataset and Countermeasures for the Universally Detection of Deepfake Audiocited by 31
- 2024RALL-E: Robust Codec Language Modeling with Chain-of-Thought Prompting for Text-to-Speech Synthesiscited by 26
- 2022Representation Selective Self-distillation and wav2vec 2.0 Feature Exploration for Spoof-aware Speaker Verificationcited by 9
- 2024HAM-TTS: Hierarchical Acoustic Modeling for Token-Based Zero-Shot Text-to-Speech with Model and Data Scalingcited by 6
- 2024Generalized Fake Audio Detection via Deep Stable Learningcited by 5
- 2024Genuine-Focused Learning using Mask AutoEncoder for Generalized Fake Audio Detectioncited by 5
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.