mlsanityData to 14 Oct 2026

Towards Supervised Performance on Speaker Verification with Self-Supervised Learning by Leveraging Large-Scale ASR Models

arXiv 2406.02285 ↗First posted 4 Jun 2024eess.AS cs.LG cs.SDcited by 4

Not on the map. Why?

Abstract (arXiv)

Recent advancements in Self-Supervised Learning (SSL) have shown promising results in Speaker Verification (SV). However, narrowing the performance gap with supervised systems remains an ongoing challenge. Several studies have observed that speech representations from large-scale ASR models contain valuable speaker information. This work explores the limitations of fine-tuning these models for SV using an SSL contrastive objective in an end-to-end approach. Then, we propose a framework to learn speaker representations in an SSL context by fine-tuning a pre-trained WavLM with a supervised loss using pseudo-labels. Initial pseudo-labels are derived from an SSL DINO-based model and are iteratively refined by clustering the model embeddings. Our method achieves 0.99% EER on VoxCeleb1-O, establishing the new state-of-the-art on self-supervised SV. As this performance is close to our supervised baseline of 0.94% EER, this contribution is a step towards supervised performance on SV with SSL.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 27 of its 35 references that are in the map. First posted on arXiv 4 Jun 2024.

Builds on

27 of 35 references.

References · 27

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  3. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  4. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  5. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  6. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  7. 2018VoxCeleb2: Deep Speaker Recognitioncited by 875
  8. 2017VoxCeleb: a large-scale speaker identification datasetcited by 732
  9. 2020ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verificationcited by 628
  10. 2015MUSAN: A Music, Speech, and Noise Corpuscited by 425
  11. 2019How to Fine-Tune BERT for Text Classification?cited by 411
  12. 2020In defence of metric learning for speaker recognitioncited by 126
15 more
  1. 2020Exploring wav2vec 2.0 on speaker verification and language identificationcited by 95
  2. 2020Self-supervised Text-independent Speaker Verification using Prototypical Momentum Contrastive Learningcited by 33
  3. 2022Why does Self-Supervised Learning for Speech Recognition Benefit Speaker Recognition?cited by 27
  4. 2018Learning Speaker Representations with Mutual Informationcited by 26
  5. 2020Augmentation adversarial training for self-supervised speaker recognitioncited by 24
  6. 2022An attention-based backend allowing efficient fine-tuning of transformer models for speaker verificationcited by 16
  7. 2022Pushing the limits of self-supervised speaker verification using regularized distillation frameworkcited by 15
  8. 2021Self-supervised Speaker Recognition with Loss-gated Learningcited by 14
  9. 2022C3-DINO: Joint Contrastive and Non-contrastive Self-Supervised Learning for Speaker Verificationcited by 12
  10. 2022Label-Efficient Self-Supervised Speaker Verification With Information Maximization and Contrastive Learningcited by 10
  11. 2022Non-Contrastive Self-supervised Learning for Utterance-Level Information Extraction from Speechcited by 10
  12. 2021What do End-to-End Speech Models Learn about Speaker, Language and Channel Information? A Layer-wise and Neuron-level Analysiscited by 9
  13. 2022Self-Supervised Speaker Verification Using Dynamic Loss-Gate and Label Correctioncited by 6
  14. 2021The DKU-DukeECE System for the Self-Supervision Speaker Verification Task of the 2021 VoxCeleb Speaker Recognition Challengecited by 3
  15. 2021The JHU submission to VoxSRC-21: Track 3cited by 1

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 1.

4 papers.

  1. 2025Recent Advances in Discrete Speech Tokens: A Reviewcited by 43
  2. 2024Exploring WavLM Back-ends for Speech Spoofing and Deepfake Detectioncited by 10
  3. 2025Identifying Speaker Information in Feed-Forward Layers of Self-Supervised Speech Transformerscited by 1
  4. 2026Self-Supervised Learning for Speaker Recognition: A study and reviewcited by 0