mlsanityData to 14 Oct 2026

Attentive Merging of Hidden Embeddings from Pre-trained Speech Model for Anti-spoofing Detection

arXiv 2406.10283 ↗First posted 12 Jun 2024cs.CL cs.SD eess.AScited by 17

Not on the map. Why?

Abstract (arXiv)

Self-supervised learning (SSL) speech representation models, trained on large speech corpora, have demonstrated effectiveness in extracting hierarchical speech embeddings through multiple transformer layers. However, the behavior of these embeddings in specific tasks remains uncertain. This paper investigates the multi-layer behavior of the WavLM model in anti-spoofing and proposes an attentive merging method to leverage the hierarchical hidden embeddings. Results demonstrate the feasibility of fine-tuning WavLM to achieve the best equal error rate (EER) of 0.65%, 3.50%, and 3.19% on the ASVspoof 2019LA, 2021LA, and 2021DF evaluation sets, respectively. Notably, We find that the early hidden transformer layers of the WavLM large model contribute significantly to anti-spoofing task, enabling computational efficiency by utilizing a partial pre-trained model.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Builds on 20 of its 33 references that are in the map. First posted on arXiv 12 Jun 2024.

Builds on

20 of 33 references.

References · 20

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  3. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  4. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  5. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 666
  6. 2020ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verificationcited by 628
  7. 2020A Survey of Convolutional Neural Networks: Analysis, Applications, and Prospectscited by 397
  8. 2021Layer-wise Analysis of a Self-supervised Speech Representation Modelcited by 298
  9. 2020End-to-end anti-spoofing with RawNet2cited by 164
  10. 2021ASVspoof 2021: accelerating progress in spoofed and deepfake speech detectioncited by 161
  11. 2022Automatic speaker verification spoofing and deepfake detection using wav2vec 2.0 and data augmentationcited by 117
  12. 2020Exploring wav2vec 2.0 on speaker verification and language identificationcited by 95
8 more
  1. 2021ASVspoof 2019: spoofing countermeasures for the detection of synthesized, converted and replayed speechcited by 58
  2. 2021Investigating self-supervised front ends for speech spoofing countermeasurescited by 46
  3. 2022The Vicomtech Audio Deepfake Detection System based on Wav2Vec2 for the 2022 ADD Challengecited by 35
  4. 2019Deep Residual Neural Networks for Audio Spoofing Detectioncited by 33
  5. 2019ASSERT: Anti-Spoofing with Squeeze-Excitation and Residual neTworkscited by 23
  6. 2023Disentangling Voice and Content with Self-Supervision for Speaker Recognitioncited by 13
  7. 2022Fully Automated End-to-End Fake Audio Detectioncited by 10
  8. 2021UR Channel-Robust Synthetic Speech Detection System for ASVspoof 2021cited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 5; 2025: 5; 2026: 7.

17 papers.

Sort:

By area:

  1. 2024DFADD: The Diffusion and Flow-Matching Based Audio Deepfake Datasetcited by 20
  2. 2024Mixture of Experts Fusion for Fake Audio Detection Using Frozen wav2vec 2.0cited by 20
  3. 2025Nes2Net: A Lightweight Nested Architecture for Foundation Model Driven Speech Anti-spoofingcited by 19
  4. 2024Speech Foundation Model Ensembles for the Controlled Singing Voice Deepfake Detection (CtrSVDD) Challenge 2024cited by 7
  5. 2024Towards Quantifying and Reducing Language Mismatch Effects in Cross-Lingual Speech Anti-Spoofingcited by 7
  6. 2025Detect All-Type Deepfake Audio: Wavelet Prompt Tuning for Enhanced Auditory Perceptioncited by 7
  7. 2026Do Audio LLMs Listen or Read? Analyzing and Mitigating Paralinguistic Failures with VoxParadoxcited by 7
  8. 2026Interpretable All-Type Audio Deepfake Detection with Audio LLMs via Frequency-Time Reinforcement Learningcited by 6
  9. 2024Passive Deepfake Detection Across Multi-modalities: A Comprehensive Surveycited by 4
  10. 2026AT-ADD: All-Type Audio Deepfake Detection Challenge Evaluation Plancited by 3
  11. 2025Speech-FT: Merging Pre-trained And Fine-Tuned Speech Representation Models For Cross-Task Generalizationcited by 2
  12. 2025QAMO: Quality-aware Multi-centroid One-class Learning For Speech Deepfake Detectioncited by 1
5 more
  1. 2025Generalizable Speech Deepfake Detection via Information Bottleneck Enhanced Adversarial Alignmentcited by 1
  2. 2026Quantizer-Aware Hierarchical Neural Codec Modeling for Speech Deepfake Detectioncited by 0
  3. 2026Dual-Granularity Orthogonal Disentanglement for Generalizable Audio Deepfake Detectioncited by 0
  4. 2026Supervised Post-training of Speech Foundation Models for Robust Adaptation in Speech Deepfake Detectioncited by 0
  5. 2026Learning as Deepfakes Evolve: RF-Prompt for Continual Audio Deepfake Detectioncited by 0