mlsanityData to 7 Oct 2026

Improved DeepFake Detection Using Whisper Features

arXiv 2306.01428 ↗First posted 2 Jun 2023cs.SD cs.LG eess.AScited by 28

Not on the map. Why?

Abstract (arXiv)

With a recent influx of voice generation methods, the threat introduced by audio DeepFake (DF) is ever-increasing. Several different detection methods have been presented as a countermeasure. Many methods are based on so-called front-ends, which, by transforming the raw audio, emphasize features crucial for assessing the genuineness of the audio sample. Our contribution contains investigating the influence of the state-of-the-art Whisper automatic speech recognition model as a DF detection front-end. We compare various combinations of Whisper and well-established front-ends by training 3 detection models (LCNN, SpecRNet, and MesoNet) on a widely used ASVspoof 2021 DF dataset and later evaluating them on the DF In-The-Wild dataset. We show that using Whisper-based features improves the detection for each model and outperforms recent results on the In-The-Wild dataset by reducing Equal Error Rate by 21%.

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Builds on 26 of its 29 references that are in the map. First posted on arXiv 2 Jun 2023.

Builds on

26 of 29 references.

References · 26

  1. 2017Attention Is All You Needcited by 59,932
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  3. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  4. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  5. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  6. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  7. 2019Generating Long Sequences with Sparse Transformerscited by 1,523
  8. 2017Self-Normalizing Neural Networkscited by 591
  9. 2018MesoNet: a Compact Facial Video Forgery Detection Networkcited by 409
  10. 2019ASVspoof 2019: Future Horizons in Spoofed and Fake Audio Detectioncited by 173
  11. 2021AASIST: Audio Anti-Spoofing using Integrated Spectro-Temporal Graph Attention Networkscited by 173
  12. 2020End-to-end anti-spoofing with RawNet2cited by 160
14 more
  1. 2021ASVspoof 2021: accelerating progress in spoofed and deepfake speech detectioncited by 157
  2. 2022Does Audio Deepfake Detection Generalize?cited by 129
  3. 2015A Light CNN for Deep Face Representation with Noisy Labelscited by 126
  4. 2022ASVspoof 2021: Towards Spoofed and Deepfake Speech Detection in the Wildcited by 118
  5. 2022Automatic speaker verification spoofing and deepfake detection using wav2vec 2.0 and data augmentationcited by 110
  6. 2021Deepfakes Generation and Detection: State-of-the-art, open challenges, countermeasures, and way forwardcited by 82
  7. 2021WaveFake: A Data Set to Facilitate Audio Deepfake Detectioncited by 78
  8. 2022Pushing the limits of raw waveform speaker recognitioncited by 49
  9. 2021Investigating self-supervised front ends for speech spoofing countermeasurescited by 43
  10. 2020Tandem Assessment of Spoofing Countermeasures and Automatic Speaker Verification: Fundamentalscited by 22
  11. 2022Fully Automated End-to-End Fake Audio Detectioncited by 10
  12. 2022Attack Agnostic Dataset: Towards Generalization and Stabilization of Audio DeepFake Detectioncited by 9
  13. 2022SpecRNet: Towards Faster and More Accessible Audio DeepFake Detectioncited by 9
  14. 2022The Vicomtech Spoofing-Aware Biometric System for the SASV Challengecited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 12; 2025: 8; 2026: 7.

28 papers.

Sort:

By area:

  1. 2024SONICS: Synthetic Or Not -- Identifying Counterfeit Songscited by 21
  2. 2025Speech DF Arena: A Leaderboard for Speech DeepFake Detection Modelscited by 20
  3. 2024Where are we in audio deepfake detection? A systematic analysis over generative and detection modelscited by 11
  4. 2024How Do Neural Spoofing Countermeasures Detect Partially Spoofed Audio?cited by 10
  5. 2024Harder or Different? Understanding Generalization of Audio Deepfake Detectioncited by 9
  6. 2024Fake Artificial Intelligence Generated Contents (FAIGC): A Survey of Theories, Detection Methods, and Opportunitiescited by 7
  7. 2024Prompt Tuning for Audio Deepfake Detection: Computationally Efficient Test-time Domain Adaptation with Limited Target Datasetcited by 7
  8. 2024Retrieval-Augmented Audio Deepfake Detectioncited by 6
  9. 2025Replay Attacks Against Audio Deepfake Detectioncited by 6
  10. 2023Whisper in Focus: Enhancing Stuttered Speech Classification with Encoder Layer Optimizationcited by 5
  11. 2024Easy, Interpretable, Effective: openSMILE for voice deepfake detectioncited by 5
  12. 2024SecureSpectra: Safeguarding Digital Identity from Deep Fake Threats via Intelligent Signaturescited by 4
16 more
  1. 2024Classification of Spontaneous and Scripted Speech for Multilingual Audiocited by 4
  2. 2024DeepFake-O-Meter v2.0: An Open Platform for DeepFake Detectioncited by 3
  3. 2025PhonemeFake: Redefining Deepfake Realism with Language-Driven Segmental Manipulation and Adaptive Bilevel Detectioncited by 3
  4. 2025Measuring the Robustness of Audio Deepfake Detection under Real-World Corruptioncited by 2
  5. 2025DeePen: Penetration Testing for Audio Deepfake Detectioncited by 1
  6. 2025IndieFake Dataset: A Benchmark Dataset for Audio Deepfake Detectioncited by 1
  7. 2026Investigating the Viability of Employing Multi-modal Large Language Models in the Context of Audio Deepfake Detectioncited by 1
  8. 2026DetectZoo: A Unified Toolkit for AI-Generated Content Detection Across Text, Audio, and Image Modalitiescited by 1
  9. 2026Teffic-Audio: Tell Fact from Fictioncited by 1
  10. 2024A New Approach to Voice Authenticitycited by 0
  11. 2025As Good as It KAN Get: High-Fidelity Audio Representationcited by 0
  12. 2025Towards Generalized Source Tracing for Codec-Based Deepfake Speechcited by 0
  13. 2026Audio Deepfake Detection in the Age of Advanced Text-to-Speech modelscited by 0
  14. 2026Probing Speaker Identity Sensitivity in Audio Deepfake Detectorscited by 0
  15. 2026Spooftral: Can Voxtral Audio-Language Model Detect Speech Spoofing?cited by 0
  16. 2026Tracing Decoder Artifacts for Compact Synthetic Speech Screeningcited by 0