# I Can Hear You: Selective Robust Training for Deepfake Audio Detection

arXiv: https://arxiv.org/abs/2411.00121 · HTML: https://mlsanity.com/paper/2411.00121/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 45 references that are in the map.
- First posted on arXiv 31 Oct 2024.

## Builds on (most cited first, 30 of 35)

- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [Intriguing properties of neural networks](https://mlsanity.com/paper/1312.6199/) (2013)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [RandAugment: Practical automated data augmentation with a reduced search space](https://mlsanity.com/paper/1909.13719/) (2019)
- [Theoretically Principled Trade-off between Robustness and Accuracy](https://mlsanity.com/paper/1901.08573/) (2019)
- [Robustness May Be at Odds with Accuracy](https://mlsanity.com/paper/1805.12152/) (2018)
- [VoxCeleb: a large-scale speaker identification dataset](https://mlsanity.com/paper/1706.08612/) (2017)
- [ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification](https://mlsanity.com/paper/2005.07143/) (2020)
- [CosyVoice: A Scalable Multilingual Zero-shot Text-to-speech Synthesizer based on Supervised Semantic Tokens](https://mlsanity.com/paper/2407.05407/) (2024)
- [NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers](https://mlsanity.com/paper/2304.09116/) (2023)
- [NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Models](https://mlsanity.com/paper/2403.03100/) (2024)
- [XTTS: a Massively Multilingual Zero-Shot Text-to-Speech Model](https://mlsanity.com/paper/2406.04904/) (2024)
- [ASVspoof 2019: Future Horizons in Spoofed and Fake Audio Detection](https://mlsanity.com/paper/1904.05441/) (2019)
- [Speaker Recognition from Raw Waveform with SincNet](https://mlsanity.com/paper/1808.00158/) (2018)
- [End-to-end anti-spoofing with RawNet2](https://mlsanity.com/paper/2011.01108/) (2020)
- [StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Models](https://mlsanity.com/paper/2306.07691/) (2023)
- [Does Audio Deepfake Detection Generalize?](https://mlsanity.com/paper/2203.16263/) (2022)
- [ASVspoof 2021: Towards Spoofed and Deepfake Speech Detection in the Wild](https://mlsanity.com/paper/2210.02437/) (2022)
- [VoiceCraft: Zero-Shot Speech Editing and Text-to-Speech in the Wild](https://mlsanity.com/paper/2403.16973/) (2024)
- [Diffusion-Based Voice Conversion with Fast Maximum Likelihood Sampling Scheme](https://mlsanity.com/paper/2109.13821/) (2021)
- [WaveFake: A Data Set to Facilitate Audio Deepfake Detection](https://mlsanity.com/paper/2111.02813/) (2021)
- [Metric Learning for Adversarial Robustness](https://mlsanity.com/paper/1909.00900/) (2019)
- [Better speech synthesis through scaling](https://mlsanity.com/paper/2305.07243/) (2023)
- [Pushing the limits of raw waveform speaker recognition](https://mlsanity.com/paper/2203.08488/) (2022)
- [AI-Synthesized Voice Detection Using Neural Vocoder Artifacts](https://mlsanity.com/paper/2304.13085/) (2023)
- [Replay and Synthetic Speech Detection with Res2net Architecture](https://mlsanity.com/paper/2010.15006/) (2020)
- [Deep Residual Neural Networks for Audio Spoofing Detection](https://mlsanity.com/paper/1907.00501/) (2019)
- [Learning Filterbanks from Raw Speech for Phone Recognition](https://mlsanity.com/paper/1711.01161/) (2017)
- [Towards Understanding and Mitigating Audio Adversarial Examples for Speaker Recognition](https://mlsanity.com/paper/2206.03393/) (2022)
- [Defending against Adversarial Audio via Diffusion Model](https://mlsanity.com/paper/2303.01507/) (2023)

## Built on by (most cited first, 13 of 13)

- [ALLM4ADD: Unlocking the Capabilities of Audio Large Language Models for Audio Deepfake Detection](https://mlsanity.com/paper/2505.11079/) (2025)
- [Deepfake Media Generation and Detection in the Generative AI Era: A Survey and Outlook](https://mlsanity.com/paper/2411.19537/) (2024)
- [Detect All-Type Deepfake Audio: Wavelet Prompt Tuning for Enhanced Auditory Perception](https://mlsanity.com/paper/2504.06753/) (2025)
- [Interpretable All-Type Audio Deepfake Detection with Audio LLMs via Frequency-Time Reinforcement Learning](https://mlsanity.com/paper/2601.02983/) (2026)
- [SHIELD: A Secure and Highly Enhanced Integrated Learning for Robust Deepfake Detection against Adversarial Attacks](https://mlsanity.com/paper/2507.13170/) (2025)
- [AT-ADD: All-Type Audio Deepfake Detection Challenge Evaluation Plan](https://mlsanity.com/paper/2604.08184/) (2026)
- [Investigating the Viability of Employing Multi-modal Large Language Models in the Context of Audio Deepfake Detection](https://mlsanity.com/paper/2601.00777/) (2026)
- [AUDETER: A Large-scale Dataset for Deepfake Audio Detection in Open Worlds](https://mlsanity.com/paper/2509.04345/) (2025)
- [E2E-VGuard: Adversarial Prevention for Production LLM-based End-To-End Speech Synthesis](https://mlsanity.com/paper/2511.07099/) (2025)
- [Variable-Length Audio Fingerprinting](https://mlsanity.com/paper/2603.23947/) (2026)
- [DF-MoE: Generalizable Deepfake Detection via Multimodal Sparse Mixture-of-Experts](https://mlsanity.com/paper/2608.23363/) (2026)
- [Robust Workflow Generation via Adversarial Learning for Audio Deepfake Detection](https://mlsanity.com/paper/2609.20063/) (2026)
- [Adaptive Compression and Targeted Perturbation: A Unified Framework for Generalized Audio Deepfake Detection](https://mlsanity.com/paper/2610.16049/) (2026)

## Abstract (arXiv)

Recent advances in AI-generated voices have intensified the challenge of detecting deepfake audio, posing risks for scams and the spread of disinformation. To tackle this issue, we establish the largest public voice dataset to date, named DeepFakeVox-HQ, comprising 1.3 million samples, including 270,000 high-quality deepfake samples from 14 diverse sources. Despite previously reported high accuracy, existing deepfake voice detectors struggle with our diversely collected dataset, and their detection success rates drop even further under realistic corruptions and adversarial attacks. We conduct a holistic investigation into factors that enhance model robustness and show that incorporating a diversified set of voice augmentations is beneficial. Moreover, we find that the best detection models often rely on high-frequency features, which are imperceptible to humans and can be easily manipulated by an attacker. To address this, we propose the F-SAT: Frequency-Selective Adversarial Training method focusing on high-frequency components. Empirical results demonstrate that using our training dataset boosts baseline model performance (without robust training) by 33%, and our robust training further improves accuracy by 7.7% on clean samples and by 29.3% on corrupted and attacked samples, over the state-of-the-art RawNet3 model.