mlsanityData to 7 Oct 2026

Temporal-Channel Modeling in Multi-head Self-Attention for Synthetic Speech Detection

arXiv 2406.17376 ↗First posted 25 Jun 2024cs.SD cs.AI eess.AScited by 24

Not on the map. Why?

Abstract (arXiv)

Recent synthetic speech detectors leveraging the Transformer model have superior performance compared to the convolutional neural network counterparts. This improvement could be due to the powerful modeling ability of the multi-head self-attention (MHSA) in the Transformer model, which learns the temporal relationship of each input token. However, artifacts of synthetic speech can be located in specific regions of both frequency channels and temporal segments, while MHSA neglects this temporal-channel dependency of the input sequence. In this work, we proposed a Temporal-Channel Modeling (TCM) module to enhance MHSA's capability for capturing temporal-channel dependencies. Experimental results on the ASVspoof 2021 show that with only 0.03M additional parameters, the TCM module can outperform the state-of-the-art system by 9.25% in EER. Further ablation study reveals that utilizing both temporal and channel information yields the most improvement for detecting synthetic speech.

Key facts. Cited by 24 papers in the map (data through 7 Oct 2026). Builds on 23 of its 31 references that are in the map. First posted on arXiv 25 Jun 2024.

Builds on

23 of 31 references.

References · 23

  1. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 589
  2. 2019ASVspoof 2019: Future Horizons in Spoofed and Fake Audio Detectioncited by 173
  3. 2021AASIST: Audio Anti-Spoofing using Integrated Spectro-Temporal Graph Attention Networkscited by 173
  4. 2020End-to-end anti-spoofing with RawNet2cited by 160
  5. 2022Does Audio Deepfake Detection Generalize?cited by 129
  6. 2020Multi-task self-supervised learning for Robust Speech Recognitioncited by 127
  7. 2022ASVspoof 2021: Towards Spoofed and Deepfake Speech Detection in the Wildcited by 118
  8. 2022Automatic speaker verification spoofing and deepfake detection using wav2vec 2.0 and data augmentationcited by 110
  9. 2021RawBoost: A Raw Data Boosting and Augmentation Method applied to Automatic Speaker Verification Anti-Spoofingcited by 64
  10. 2022Speech Emotion Recognition using Self-Supervised Featurescited by 52
  11. 2019STC Antispoofing Systems for the ASVspoof2019 Challengecited by 51
  12. 2020Replay and Synthetic Speech Detection with Res2net Architecturecited by 38
11 more
  1. 2022Bridging the Gap Between Vision Transformers and Convolutional Neural Networks on Small Datasetscited by 28
  2. 2020Light Convolutional Neural Network with Feature Genuinization for Detection of Synthetic Speech Attackscited by 23
  3. 2021Channel-wise Gated Res2Net: Towards Robust Detection of Synthetic Speech Attackscited by 18
  4. 2020An explainability study of the constant Q cepstral coefficient spoofing countermeasure for automatic speaker verificationcited by 13
  5. 2013The BOSARIS Toolkit: Theory, Algorithms and Code for Surviving the New DCFcited by 12
  6. 2023The defender's perspective on automatic speaker verification: An overviewcited by 8
  7. 2023HM-Conformer: A Conformer-based audio deepfake detection system with hierarchical pooling and multi-level classification token aggregation methodscited by 8
  8. 2022Estimation of speaker age and height from speech signal using bi-encoder transformer mixture modelcited by 6
  9. 2022Voice Spoofing Countermeasures: Taxonomy, State-of-the-art, experimental analysis of generalizability, open challenges, and the way forwardcited by 4
  10. 2023Emphasized Non-Target Speaker Knowledge in Knowledge Distillation for Automatic Speaker Verificationcited by 2
  11. 2021Efficient Attention Branch Network with Combined Loss Function for Automatic Speaker Verification Spoof Detectioncited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 15; 2026: 7.

24 papers.

Sort:

By area:

  1. 2024XLSR-Mamba: A Dual-Column Bidirectional State Space Model for Spoofing Attack Detectioncited by 27
  2. 2025Speech DF Arena: A Leaderboard for Speech DeepFake Detection Modelscited by 20
  3. 2025Nes2Net: A Lightweight Nested Architecture for Foundation Model Driven Speech Anti-spoofingcited by 17
  4. 2024Speech Foundation Model Ensembles for the Controlled Singing Voice Deepfake Detection (CtrSVDD) Challenge 2024cited by 7
  5. 2025Replay Attacks Against Audio Deepfake Detectioncited by 6
  6. 2025Fake-Mamba: Real-Time Speech Deepfake Detection Using Bidirectional Mamba as Self-Attention's Alternativecited by 6
  7. 2025Unveiling Audio Deepfake Origins: A Deep Metric learning And Conformer Network Approach With Ensemble Fusioncited by 5
  8. 2026Do Compact SSL Backbones Matter for Audio Deepfake Detection? A Controlled Study with RAPTORcited by 4
  9. 2025Bona fide Cross Testing Reveals Weak Spot in Audio Deepfake Detection Systemscited by 3
  10. 2025Pushing the Performance of Synthetic Speech Detection with Kolmogorov-Arnold Networks and Self-Supervised Learning Modelscited by 2
  11. 2025Addressing Gradient Misalignment in Data-Augmented Training for Robust Speech Deepfake Detectioncited by 2
  12. 2025XMAD-Bench: Cross-Domain Multilingual Audio Deepfake Benchmarkcited by 1
12 more
  1. 2025Fake Speech Wild: Detecting Deepfake Speech on Social Media Platformcited by 1
  2. 2025QAMO: Quality-aware Multi-centroid One-class Learning For Speech Deepfake Detectioncited by 1
  3. 2025XLSR-Kanformer: A KAN-Intergrated model for Synthetic Speech Detectioncited by 1
  4. 2025SONAR: Spectral-Contrastive Audio Residuals for Generalizable Deepfake Detectioncited by 1
  5. 2026Teffic-Audio: Tell Fact from Fictioncited by 1
  6. 2025Acoustic scattering AI for non-invasive object classifications: A case study on hair assessmentcited by 0
  7. 2025Can Current Detectors Catch Face-to-Voice Deepfake Attacks?cited by 0
  8. 2026HyperPotter: Spell the Charm of High-Order Interactions in Audio Deepfake Detectioncited by 0
  9. 2026ICLAD: In-Context Learning with Comparison-Guidance for Audio Deepfake Detectioncited by 0
  10. 2026Time-Frequency Consistency Learning for Robust Speech Deepfake Detectioncited by 0
  11. 2026Domain-Adaptive Dual-Gating Mixture of Experts for Generalizable Speech Deepfake Detectioncited by 0
  12. 2026GLAD: Global-Local Adaptive Detector for Robust Speech Deepfake Detectioncited by 0