# Self-Attention and Hybrid Features for Replay and Deep-Fake Audio Detection arXiv: https://arxiv.org/abs/2401.05614 ยท HTML: https://mlsanity.com/paper/2401.05614/ ## Key facts - Cited by 1 papers in the map (data through 20 Oct 2026). - Placed in Anti-spoofing and adversarial audio attacks, one of 11 directions in Speech and audio language models. - Builds on 29 of its 61 references that are in the map. - First posted on arXiv 11 Jan 2024. ## Placed in - Speech and audio language models / [Anti-spoofing and adversarial audio attacks](https://mlsanity.com/speech/anti-spoofing-and-adversarial-audio-attacks/) ## Builds on (most cited first, 29 of 29) - [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017) - [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015) - [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016) - [Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictions](https://mlsanity.com/paper/1712.05884/) (2017) - [Exploring Self-attention for Image Recognition](https://mlsanity.com/paper/2004.13621/) (2020) - [Focal Self-attention for Local-Global Interactions in Vision Transformers](https://mlsanity.com/paper/2107.00641/) (2021) - [AASIST: Audio Anti-Spoofing using Integrated Spectro-Temporal Graph Attention Networks](https://mlsanity.com/paper/2110.01200/) (2021) - [ASVspoof 2019: Future Horizons in Spoofed and Fake Audio Detection](https://mlsanity.com/paper/1904.05441/) (2019) - [Towards End-to-End Prosody Transfer for Expressive Speech Synthesis with Tacotron](https://mlsanity.com/paper/1803.09047/) (2018) - [ASVspoof 2021: accelerating progress in spoofed and deepfake speech detection](https://mlsanity.com/paper/2109.00537/) (2021) - [End-to-end anti-spoofing with RawNet2](https://mlsanity.com/paper/2011.01108/) (2020) - [ASVspoof 2021: Towards Spoofed and Deepfake Speech Detection in the Wild](https://mlsanity.com/paper/2210.02437/) (2022) - [Self-Attention Attribution: Interpreting Information Interactions Inside Transformer](https://mlsanity.com/paper/2004.11207/) (2020) - [ADD 2022: the First Audio Deep Synthesis Detection Challenge](https://mlsanity.com/paper/2202.08433/) (2022) - [Beyond Self-attention: External Attention using Two Linear Layers for Visual Tasks](https://mlsanity.com/paper/2105.02358/) (2021) - [LEAF: A Learnable Frontend for Audio Classification](https://mlsanity.com/paper/2101.08596/) (2021) - [ASVspoof 2019: spoofing countermeasures for the detection of synthesized, converted and replayed speech](https://mlsanity.com/paper/2102.05889/) (2021) - [On the Integration of Self-Attention and Convolution](https://mlsanity.com/paper/2111.14556/) (2021) - [STC Antispoofing Systems for the ASVspoof2019 Challenge](https://mlsanity.com/paper/1904.05576/) (2019) - [A Comparative Study on Recent Neural Spoofing Countermeasures for Synthetic Speech Detection](https://mlsanity.com/paper/2103.11326/) (2021) - [Audio Transformers](https://mlsanity.com/paper/2105.00335/) (2021) - [Tandem Assessment of Spoofing Countermeasures and Automatic Speaker Verification: Fundamentals](https://mlsanity.com/paper/2007.05979/) (2020) - [t-DCF: a Detection Cost Function for the Tandem Assessment of Spoofing Countermeasures and Automatic Speaker Verification](https://mlsanity.com/paper/1804.09618/) (2018) - [Improved RawNet with Feature Map Scaling for Text-independent Speaker Verification using Raw Waveforms](https://mlsanity.com/paper/2004.00526/) (2020) - [Partially Fake Audio Detection by Self-attention-based Fake Span Discovery](https://mlsanity.com/paper/2202.06684/) (2022) - [S-vectors and TESA: Speaker Embeddings and a Speaker Authenticator Based on Transformer Encoder](https://mlsanity.com/paper/2008.04659/) (2020) - [FastAudio: A Learnable Audio Front-End for Spoof Speech Detection](https://mlsanity.com/paper/2109.02774/) (2021) - [Audio Deepfake Detection Based on a Combination of F0 Information and Real Plus Imaginary Spectrogram Features](https://mlsanity.com/paper/2208.01214/) (2022) - [Dataset artefacts in anti-spoofing systems: a case study on the ASVspoof 2017 benchmark](https://mlsanity.com/paper/2010.07913/) (2020) ## Built on by (most cited first, 1 of 1) - [Bridging Attribution and Open-Set Detection using Graph-Augmented Instance Learning in Synthetic Speech](https://mlsanity.com/paper/2601.07064/) (2026) ## Abstract (arXiv) Due to the successful application of deep learning, audio spoofing detection has made significant progress. Spoofed audio with speech synthesis or voice conversion can be well detected by many countermeasures. However, an automatic speaker verification system is still vulnerable to spoofing attacks such as replay or Deep-Fake audio. Deep-Fake audio means that the spoofed utterances are generated using text-to-speech (TTS) and voice conversion (VC) algorithms. Here, we propose a novel framework based on hybrid features with the self-attention mechanism. It is expected that hybrid features can be used to get more discrimination capacity. Firstly, instead of only one type of conventional feature, deep learning features and Mel-spectrogram features will be extracted by two parallel paths: convolution neural networks and a short-time Fourier transform (STFT) followed by Mel-frequency. Secondly, features will be concatenated by a max-pooling layer. Thirdly, there is a Self-attention mechanism for focusing on essential elements. Finally, ResNet and a linear layer are built to get the results. Experimental results reveal that the hybrid features, compared with conventional features, can cover more details of an utterance. We achieve the best Equal Error Rate (EER) of 9.67% in the physical access (PA) scenario and 8.94% in the Deep fake task on the ASVspoof 2021 dataset. Compared with the best baseline system, the proposed approach improves by 74.60% and 60.05%, respectively.