# Improved DeepFake Detection Using Whisper Features

arXiv: https://arxiv.org/abs/2306.01428 · HTML: https://mlsanity.com/paper/2306.01428/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 29 references that are in the map.
- First posted on arXiv 2 Jun 2023.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Towards Deep Learning Models Resistant to Adversarial Attacks](https://mlsanity.com/paper/1706.06083/) (2017)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Generating Long Sequences with Sparse Transformers](https://mlsanity.com/paper/1904.10509/) (2019)
- [Self-Normalizing Neural Networks](https://mlsanity.com/paper/1706.02515/) (2017)
- [MesoNet: a Compact Facial Video Forgery Detection Network](https://mlsanity.com/paper/1809.00888/) (2018)
- [AASIST: Audio Anti-Spoofing using Integrated Spectro-Temporal Graph Attention Networks](https://mlsanity.com/paper/2110.01200/) (2021)
- [ASVspoof 2019: Future Horizons in Spoofed and Fake Audio Detection](https://mlsanity.com/paper/1904.05441/) (2019)
- [End-to-end anti-spoofing with RawNet2](https://mlsanity.com/paper/2011.01108/) (2020)
- [ASVspoof 2021: accelerating progress in spoofed and deepfake speech detection](https://mlsanity.com/paper/2109.00537/) (2021)
- [Does Audio Deepfake Detection Generalize?](https://mlsanity.com/paper/2203.16263/) (2022)
- [A Light CNN for Deep Face Representation with Noisy Labels](https://mlsanity.com/paper/1511.02683/) (2015)
- [ASVspoof 2021: Towards Spoofed and Deepfake Speech Detection in the Wild](https://mlsanity.com/paper/2210.02437/) (2022)
- [Automatic speaker verification spoofing and deepfake detection using wav2vec 2.0 and data augmentation](https://mlsanity.com/paper/2202.12233/) (2022)
- [Deepfakes Generation and Detection: State-of-the-art, open challenges, countermeasures, and way forward](https://mlsanity.com/paper/2103.00484/) (2021)
- [WaveFake: A Data Set to Facilitate Audio Deepfake Detection](https://mlsanity.com/paper/2111.02813/) (2021)
- [Pushing the limits of raw waveform speaker recognition](https://mlsanity.com/paper/2203.08488/) (2022)
- [Investigating self-supervised front ends for speech spoofing countermeasures](https://mlsanity.com/paper/2111.07725/) (2021)
- [Tandem Assessment of Spoofing Countermeasures and Automatic Speaker Verification: Fundamentals](https://mlsanity.com/paper/2007.05979/) (2020)
- [Fully Automated End-to-End Fake Audio Detection](https://mlsanity.com/paper/2208.09618/) (2022)
- [Attack Agnostic Dataset: Towards Generalization and Stabilization of Audio DeepFake Detection](https://mlsanity.com/paper/2206.13979/) (2022)
- [SpecRNet: Towards Faster and More Accessible Audio DeepFake Detection](https://mlsanity.com/paper/2210.06105/) (2022)
- [The Vicomtech Spoofing-Aware Biometric System for the SASV Challenge](https://mlsanity.com/paper/2204.01399/) (2022)

## Built on by (most cited first, 29 of 29)

- [SONICS: Synthetic Or Not -- Identifying Counterfeit Songs](https://mlsanity.com/paper/2408.14080/) (2024)
- [Speech DF Arena: A Leaderboard for Speech DeepFake Detection Models](https://mlsanity.com/paper/2509.02859/) (2025)
- [Where are we in audio deepfake detection? A systematic analysis over generative and detection models](https://mlsanity.com/paper/2410.04324/) (2024)
- [How Do Neural Spoofing Countermeasures Detect Partially Spoofed Audio?](https://mlsanity.com/paper/2406.02483/) (2024)
- [Harder or Different? Understanding Generalization of Audio Deepfake Detection](https://mlsanity.com/paper/2406.03512/) (2024)
- [Fake Artificial Intelligence Generated Contents (FAIGC): A Survey of Theories, Detection Methods, and Opportunities](https://mlsanity.com/paper/2405.00711/) (2024)
- [Retrieval-Augmented Audio Deepfake Detection](https://mlsanity.com/paper/2404.13892/) (2024)
- [Prompt Tuning for Audio Deepfake Detection: Computationally Efficient Test-time Domain Adaptation with Limited Target Dataset](https://mlsanity.com/paper/2410.09869/) (2024)
- [Replay Attacks Against Audio Deepfake Detection](https://mlsanity.com/paper/2505.14862/) (2025)
- [Whisper in Focus: Enhancing Stuttered Speech Classification with Encoder Layer Optimization](https://mlsanity.com/paper/2311.05203/) (2023)
- [Easy, Interpretable, Effective: openSMILE for voice deepfake detection](https://mlsanity.com/paper/2408.15775/) (2024)
- [SecureSpectra: Safeguarding Digital Identity from Deep Fake Threats via Intelligent Signatures](https://mlsanity.com/paper/2407.00913/) (2024)
- [Classification of Spontaneous and Scripted Speech for Multilingual Audio](https://mlsanity.com/paper/2412.11896/) (2024)
- [DeepFake-O-Meter v2.0: An Open Platform for DeepFake Detection](https://mlsanity.com/paper/2404.13146/) (2024)
- [PhonemeFake: Redefining Deepfake Realism with Language-Driven Segmental Manipulation and Adaptive Bilevel Detection](https://mlsanity.com/paper/2506.22783/) (2025)
- [Measuring the Robustness of Audio Deepfake Detection under Real-World Corruption](https://mlsanity.com/paper/2503.17577/) (2025)
- [DeePen: Penetration Testing for Audio Deepfake Detection](https://mlsanity.com/paper/2502.20427/) (2025)
- [IndieFake Dataset: A Benchmark Dataset for Audio Deepfake Detection](https://mlsanity.com/paper/2506.19014/) (2025)
- [Investigating the Viability of Employing Multi-modal Large Language Models in the Context of Audio Deepfake Detection](https://mlsanity.com/paper/2601.00777/) (2026)
- [DetectZoo: A Unified Toolkit for AI-Generated Content Detection Across Text, Audio, and Image Modalities](https://mlsanity.com/paper/2606.04205/) (2026)
- [Teffic-Audio: Tell Fact from Fiction](https://mlsanity.com/paper/2607.28351/) (2026)
- [A New Approach to Voice Authenticity](https://mlsanity.com/paper/2402.06304/) (2024)
- [As Good as It KAN Get: High-Fidelity Audio Representation](https://mlsanity.com/paper/2503.02585/) (2025)
- [Towards Generalized Source Tracing for Codec-Based Deepfake Speech](https://mlsanity.com/paper/2506.07294/) (2025)
- [Audio Deepfake Detection in the Age of Advanced Text-to-Speech models](https://mlsanity.com/paper/2601.20510/) (2026)
- [Probing Speaker Identity Sensitivity in Audio Deepfake Detectors](https://mlsanity.com/paper/2607.21820/) (2026)
- [Spooftral: Can Voxtral Audio-Language Model Detect Speech Spoofing?](https://mlsanity.com/paper/2609.28713/) (2026)
- [Tracing Decoder Artifacts for Compact Synthetic Speech Screening](https://mlsanity.com/paper/2609.32050/) (2026)
- [Adaptive Compression and Targeted Perturbation: A Unified Framework for Generalized Audio Deepfake Detection](https://mlsanity.com/paper/2610.16049/) (2026)

## Abstract (arXiv)

With a recent influx of voice generation methods, the threat introduced by audio DeepFake (DF) is ever-increasing. Several different detection methods have been presented as a countermeasure. Many methods are based on so-called front-ends, which, by transforming the raw audio, emphasize features crucial for assessing the genuineness of the audio sample. Our contribution contains investigating the influence of the state-of-the-art Whisper automatic speech recognition model as a DF detection front-end. We compare various combinations of Whisper and well-established front-ends by training 3 detection models (LCNN, SpecRNet, and MesoNet) on a widely used ASVspoof 2021 DF dataset and later evaluating them on the DF In-The-Wild dataset. We show that using Whisper-based features improves the detection for each model and outperforms recent results on the In-The-Wild dataset by reducing Equal Error Rate by 21%.