mlsanityData to 7 Oct 2026

Fine-tuning wav2vec2 for speaker recognition

arXiv 2109.15053 ↗First posted 30 Sep 2021cs.SD cs.LG eess.AScited by 35

Not on the map. Why?

Abstract (arXiv)

This paper explores applying the wav2vec2 framework to speaker recognition instead of speech recognition. We study the effectiveness of the pre-trained weights on the speaker recognition task, and how to pool the wav2vec2 output sequence into a fixed-length speaker embedding. To adapt the framework to speaker recognition, we propose a single-utterance classification variant with CE or AAM softmax loss, and an utterance-pair classification variant with BCE loss. Our best performing variant, w2v2-aam, achieves a 1.88% EER on the extended voxceleb1 test set compared to 1.69% EER with an ECAPA-TDNN baseline. Code is available at https://github.com/nikvaessen/w2v2-speaker.

Key facts. Cited by 35 papers in the map (data through 7 Oct 2026). Builds on 22 of its 29 references that are in the map. First posted on arXiv 30 Sep 2021.

Builds on

22 of 29 references.

References · 22

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  4. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  5. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  6. 2019XLNet: Generalized Autoregressive Pretraining for Language Understandingcited by 2,801
  7. 2018Group Normalizationcited by 1,514
  8. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  9. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
  10. 2016Deep Networks with Stochastic Depthcited by 973
  11. 2018VoxCeleb2: Deep Speaker Recognitioncited by 862
  12. 2017VoxCeleb: a large-scale speaker identification datasetcited by 714
10 more
  1. 2015Cyclical Learning Rates for Training Neural Networkscited by 627
  2. 2020ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verificationcited by 606
  3. 2019Reducing Transformer Depth on Demand with Structured Dropoutcited by 434
  4. 2021SpeechBrain: A General-Purpose Speech Toolkitcited by 404
  5. 2021Emotion Recognition from Speech Using Wav2vec 2.0 Embeddingscited by 158
  6. 2020Exploring wav2vec 2.0 on speaker verification and language identificationcited by 93
  7. 2021LeBenchmark: A Reproducible Framework for Assessing Self-Supervised Representation Learning from Speechcited by 46
  8. 2019Large Margin Softmax Loss for Speaker Verificationcited by 26
  9. 2021Improved Language Identification Through Cross-Lingual Self-Supervised Learningcited by 20
  10. 2021The Role of Phonetic Units in Speech Emotion Recognitioncited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 1; 2022: 9; 2023: 11; 2024: 6; 2025: 6; 2026: 2.

35 papers.

Sort:

  1. 2021A Fine-tuned Wav2vec 2.0/HuBERT Benchmark For Speech Emotion Recognition, Speaker Verification and Spoken Language Understandingcited by 82
  2. 2022Towards Understanding and Mitigating Audio Adversarial Examples for Speaker Recognitioncited by 22
  3. 2023LeBenchmark 2.0: a Standardized, Replicable and Enhanced Framework for Self-supervised Representations of French Speechcited by 17
  4. 2023Exploring Self-supervised Pre-trained ASR Models For Dysarthric and Elderly Speech Recognitioncited by 14
  5. 2023Audio-Visual Deception Detection: DOLOS Dataset and Parameter-Efficient Crossmodal Learningcited by 14
  6. 2023Wav2vec-based Detection and Severity Level Classification of Dysarthria from Speechcited by 14
  7. 2024Self-supervised ASR Models and Features For Dysarthric and Elderly Speech Recognitioncited by 9
  8. 2024SLIM: Style-Linguistics Mismatch Model for Generalized Audio Deepfake Detectioncited by 8
  9. 2023Leveraging Semantic Information for Efficient Self-Supervised Emotion Recognition with Audio-Textual Distilled Modelscited by 6
  10. 2022Speech Emotion: Investigating Model Representations, Multi-Task Learning and Knowledge Distillationcited by 5
  11. 2022Robust Speaker Recognition with Transformers Using wav2vec 2.0cited by 4
  12. 2022Large-scale learning of generalised representations for speaker recognitioncited by 4
23 more
  1. 2024ELP-Adapters: Parameter Efficient Adapter Tuning for Various Speech Processing Taskscited by 3
  2. 2025Efficient Finetuning for Dimensional Speech Emotion Recognition in the Age of Transformerscited by 3
  3. 2026Mega-ASR: Towards In-the-wild^2 Speech Recognition via Scaling up Real-world Acoustic Simulationcited by 3
  4. 2023Residual Information in Deep Speaker Embedding Architecturescited by 2
  5. 2023Speaker and Language Change Detection using Wav2vec2 and Whispercited by 2
  6. 2023Investigation of Self-supervised Pre-trained Models for Classification of Voice Quality from Speech and Neck Surface Accelerometer Signalscited by 2
  7. 2024Layer-aware TDNN: Speaker Recognition Using Multi-Layer Features from Pre-Trained Modelscited by 2
  8. 2025Pushing the Performance of Synthetic Speech Detection with Kolmogorov-Arnold Networks and Self-Supervised Learning Modelscited by 2
  9. 2022Training speaker recognition systems with limited datacited by 1
  10. 2022Fine-tuning Wav2vec for Vocal-burst Emotion Recognitioncited by 1
  11. 2022Universal speaker recognition encoders for different speech segments durationcited by 1
  12. 2023Unsupervised speech intelligibility assessment with utterance level alignment distance between teacher and learner Wav2Vec-2.0 representationscited by 1
  13. 2023Toward Leveraging Pre-Trained Self-Supervised Frontends for Automatic Singing Voice Understanding Tasks: Three Case Studiescited by 1
  14. 2023An Effective Transformer-based Contextual Model and Temporal Gate Pooling for Speaker Identificationcited by 1
  15. 2024Deep Insights into Cognitive Decline: A Survey of Leveraging Non-Intrusive Modalities with Deep Learning Techniquescited by 1
  16. 2025XLSR-Kanformer: A KAN-Intergrated model for Synthetic Speech Detectioncited by 1
  17. 2025Rethinking Leveraging Pre-Trained Multi-Layer Representations for Speaker Verificationcited by 1
  18. 2022Application of Knowledge Distillation to Multi-task Speech Representation Learningcited by 0
  19. 2022Dynamic Kernels and Channel Attention for Low Resource Speaker Verificationcited by 0
  20. 2024Memory-Efficient Training for Deep Speaker Embedding Learning in Speaker Verificationcited by 0
  21. 2025Speaker Fuzzy Fingerprints: Benchmarking Text-Based Identification in Multiparty Dialoguescited by 0
  22. 2025Dialect Identification Using Resource-Efficient Fine-Tuning Approachescited by 0
  23. 2026Looking and Listening Inside and Outside: Multimodal Artificial Intelligence Systems for Driver Safety Assessment and Intelligent Vehicle Decision-Makingcited by 0