mlsanityData to 20 Oct 2026

Self-Attention and Hybrid Features for Replay and Deep-Fake Audio Detection

arXiv 2401.05614 ↗First posted 11 Jan 2024cs.SD cs.MM eess.AScited by 1

On the map: Speech and audio language models → Anti-spoofing and adversarial audio attacks

Abstract (arXiv)

Due to the successful application of deep learning, audio spoofing detection has made significant progress. Spoofed audio with speech synthesis or voice conversion can be well detected by many countermeasures. However, an automatic speaker verification system is still vulnerable to spoofing attacks such as replay or Deep-Fake audio. Deep-Fake audio means that the spoofed utterances are generated using text-to-speech (TTS) and voice conversion (VC) algorithms. Here, we propose a novel framework based on hybrid features with the self-attention mechanism. It is expected that hybrid features can be used to get more discrimination capacity. Firstly, instead of only one type of conventional feature, deep learning features and Mel-spectrogram features will be extracted by two parallel paths: convolution neural networks and a short-time Fourier transform (STFT) followed by Mel-frequency. Secondly, features will be concatenated by a max-pooling layer. Thirdly, there is a Self-attention mechanism for focusing on essential elements. Finally, ResNet and a linear layer are built to get the results. Experimental results reveal that the hybrid features, compared with conventional features, can cover more details of an utterance. We achieve the best Equal Error Rate (EER) of 9.67% in the physical access (PA) scenario and 8.94% in the Deep fake task on the ASVspoof 2021 dataset. Compared with the best baseline system, the proposed approach improves by 74.60% and 60.05%, respectively.

Key facts. Cited by 1 paper in the map (data through 20 Oct 2026). Placed in Anti-spoofing and adversarial audio attacks, one of 11 directions in Speech and audio language models. Builds on 29 of its 61 references that are in the map. First posted on arXiv 11 Jan 2024.

Builds on

29 of 61 references.

Closest ancestors · 2

Same direction, newest first

  1. 2022Partially Fake Audio Detection by Self-attention-based Fake Span Discovery14 Feb
  2. 2021ASVspoof 2021: accelerating progress in spoofed and deepfake speech detection1 Sep

Other references · 27

  1. 2017Attention Is All You Needcited by 61,208
  2. 2015Deep Residual Learning for Image Recognitioncited by 52,466
  3. 2016Densely Connected Convolutional Networkscited by 6,756
  4. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 1,023
  5. 2020Exploring Self-attention for Image Recognitioncited by 314
  6. 2021Focal Self-attention for Local-Global Interactions in Vision Transformerscited by 291
  7. 2021AASIST: Audio Anti-Spoofing using Integrated Spectro-Temporal Graph Attention Networkscited by 238
  8. 2019ASVspoof 2019: Future Horizons in Spoofed and Fake Audio Detectioncited by 235
  9. 2018Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotroncited by 232
  10. 2020End-to-end anti-spoofing with RawNet2cited by 196
  11. 2022ASVspoof 2021: Towards Spoofed and Deepfake Speech Detection in the Wildcited by 151
  12. 2020Self-Attention Attribution: Interpreting Information Interactions Inside Transformercited by 127
15 more
  1. 2022ADD 2022: the First Audio Deep Synthesis Detection Challengecited by 124
  2. 2021Beyond Self-attention: External Attention using Two Linear Layers for Visual Taskscited by 110
  3. 2021LEAF: A Learnable Frontend for Audio Classificationcited by 79
  4. 2021ASVspoof 2019: spoofing countermeasures for the detection of synthesized, converted and replayed speechcited by 78
  5. 2021On the Integration of Self-Attention and Convolutioncited by 75
  6. 2019STC Antispoofing Systems for the ASVspoof2019 Challengecited by 72
  7. 2021A Comparative Study on Recent Neural Spoofing Countermeasures for Synthetic Speech Detectioncited by 71
  8. 2021Audio Transformerscited by 45
  9. 2020Tandem Assessment of Spoofing Countermeasures and Automatic Speaker Verification: Fundamentalscited by 37
  10. 2018t-DCF: a Detection Cost Function for the Tandem Assessment of Spoofing Countermeasures and Automatic Speaker Verificationcited by 30
  11. 2020Improved RawNet with Feature Map Scaling for Text-independent Speaker Verification using Raw Waveformscited by 24
  12. 2020S-vectors and TESA: Speaker Embeddings and a Speaker Authenticator Based on Transformer Encodercited by 13
  13. 2021FastAudio: A Learnable Audio Front-End for Spoof Speech Detectioncited by 10
  14. 2022Audio Deepfake Detection Based on a Combination of F0 Information and Real Plus Imaginary Spectrogram Featurescited by 10
  15. 2020Dataset artefacts in anti-spoofing systems: a case study on the ASVspoof 2017 benchmarkcited by 9

Built on by

Cited by year (2026 to 20 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Bridging Attribution and Open-Set Detection using Graph-Augmented Instance Learning in Synthetic Speechcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2022Audio Anti-spoofing Using a Simple Attention Module and Joint Optimization Based on Additive Angular Margin Loss and Meta-learning
  2. 2025When Fine-Tuning is Not Enough: Lessons from HSAD on Hybrid and Adversarial Audio Spoof Detection
  3. 2024An RFP dataset for Real, Fake, and Partially fake audio detection
  4. 2024Toward Improving Synthetic Audio Spoofing Detection Robustness via Meta-Learning and Disentangled Training With Adversarial Examples
  5. 2022Representation Selective Self-distillation and wav2vec 2.0 Feature Exploration for Spoof-aware Speaker Verification
  6. 2022Does Audio Deepfake Detection Generalize?
  7. 2022ConvNeXt Based Neural Network for Audio Anti-Spoofing
  8. 2021Physiological-Physical Feature Fusion for Automatic Voice Spoofing Detection ↗