mlsanityData to 14 Oct 2026

Why does Self-Supervised Learning for Speech Recognition Benefit Speaker Recognition?

arXiv 2204.12765 ↗First posted 27 Apr 2022cs.CL cs.SD eess.AScited by 27

Not on the map. Why?

Abstract (arXiv)

Recently, self-supervised learning (SSL) has demonstrated strong performance in speaker recognition, even if the pre-training objective is designed for speech recognition. In this paper, we study which factor leads to the success of self-supervised learning on speaker-related tasks, e.g. speaker verification (SV), through a series of carefully designed experiments. Our empirical results on the Voxceleb-1 dataset suggest that the benefit of SSL to SV task is from a combination of mask speech prediction loss, data scale, and model size, while the SSL quantizer has a minor impact. We further employ the integrated gradients attribution method and loss landscape visualization to understand the effectiveness of self-supervised learning for speaker recognition performance.

Key facts. Cited by 27 papers in the map (data through 14 Oct 2026). Builds on 22 of its 27 references that are in the map. First posted on arXiv 27 Apr 2022.

Builds on

22 of 27 references.

References · 22

  1. 2017Neural Discrete Representation Learningcited by 4,712
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  3. 2017Axiomatic Attribution for Deep Networkscited by 3,384
  4. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  5. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  6. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,380
  7. 2017Visualizing the Loss Landscape of Neural Netscited by 1,130
  8. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 666
  9. 2020ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verificationcited by 628
  10. 2019Res2Net: A New Multi-scale Backbone Architecturecited by 470
  11. 2019Libri-Light: A Benchmark for ASR with Limited or No Supervisioncited by 467
  12. 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 463
10 more
  1. 2021GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audiocited by 399
  2. 2022Self-supervised Learning with Random-projection Quantizer for Speech Recognitioncited by 162
  3. 2021BigSSL: Exploring the Frontier of Large-Scale Semi-Supervised Learning for Automatic Speech Recognitioncited by 88
  4. 2021Large-scale Self-Supervised Speech Representation Learning for Automatic Speaker Verificationcited by 78
  5. 2019Visualizing and Understanding the Effectiveness of BERTcited by 67
  6. 2021UniSpeech-SAT: Universal Speech Representation Learning with Speaker Aware Pre-Trainingcited by 45
  7. 2020The IDLAB VoxSRC-20 Submission: Large Margin Fine-Tuning and Quality-Aware Score Calibration in DNN Based Speaker Verificationcited by 23
  8. 2021The SpeakIn System for VoxCeleb Speaker Recognition Challange 2021cited by 20
  9. 2021Self-Supervised Learning for speech recognition with Intermediate layer supervisioncited by 15
  10. 2020ResNeXt and Res2Net Structures for Speaker Verificationcited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 4; 2023: 4; 2024: 8; 2025: 7; 2026: 4.

27 papers.

Sort:

  1. 2022Exploration of A Self-Supervised Speech Model: A Study on Emotional Corporacited by 31
  2. 2022Evidence of Vocal Tract Articulation in Self-Supervised Learning of Speechcited by 31
  3. 2024Coding Speech through Vocal Tract Kinematicscited by 18
  4. 2023Zero-shot text-to-speech synthesis conditioned using self-supervised speech representation modelcited by 10
  5. 2022On the Use of Semantically-Aligned Speech Representations for Spoken Language Understandingcited by 7
  6. 2022Speech separation with large-scale self-supervised learningcited by 5
  7. 2023Decoding Emotions: A comprehensive Multilingual Study of Speech Models for Speech Emotion Recognitioncited by 4
  8. 2024Noise-robust zero-shot text-to-speech synthesis conditioned on self-supervised speech-representation model with adapterscited by 4
  9. 2024Towards Supervised Performance on Speaker Verification with Self-Supervised Learning by Leveraging Large-Scale ASR Modelscited by 4
  10. 2025Layer-wise Investigation of Large-Scale Self-Supervised Music Representation Modelscited by 4
  11. 2023Improving Transformer-based Networks With Locality For Automatic Speaker Verificationcited by 3
  12. 2025Voice Impression Control in Zero-Shot TTScited by 3
15 more
  1. 2023Semantic enrichment towards efficient speech representationscited by 2
  2. 2024Lightweight Zero-shot Text-to-Speech with Mixture of Adapterscited by 2
  3. 2024Layer-aware TDNN: Speaker Recognition Using Multi-Layer Features from Pre-Trained Modelscited by 2
  4. 2024CA-SSLR: Condition-Aware Self-Supervised Learning Representation for Generalized Speech Processingcited by 2
  5. 2025Multi-interaction TTS toward professional recording reproductioncited by 2
  6. 2025DAFMSVC: One-Shot Singing Voice Conversion with Dual Attention Mechanism and Flow Matchingcited by 2
  7. 2025Transcribe, Translate, or Transliterate: An Investigation of Intermediate Representations in Spoken Language Modelscited by 2
  8. 2024A dual task learning approach to fine-tune a multilingual semantic speech encoder for Spoken Language Understandingcited by 1
  9. 2025Singing Voice Conversion with Accompaniment Using Self-Supervised Representation-Based Melody Featurescited by 1
  10. 2025Training Articulatory Inversion Models for Interspeaker Consistencycited by 1
  11. 2026Investigation for Relative Voice Impression Estimationcited by 1
  12. 2026Beyond Short Segments : Expanding Speaker Embeddings with Vector Archivescited by 1
  13. 2024An Empirical Analysis of Speech Self-Supervised Learning at Multiple Resolutionscited by 0
  14. 2026Self-Supervised Learning for Speaker Recognition: A study and reviewcited by 0
  15. 2026LASE: Language-Adversarial Speaker Encoding for Indic Cross-Script Identity Preservationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Self-supervised Reflective Learning through Self-distillation and Online Clustering for Speaker Representation Learning
  2. 2022Deep versus Wide: An Analysis of Student Architectures for Task-Agnostic Knowledge Distillation of Self-Supervised Speech Models
  3. 2023One-Step Knowledge Distillation and Fine-Tuning in Using Large Pre-Trained Self-Supervised Learning Models for Speaker Verification
  4. 2023Self-Distillation Prototypes Network: Learning Robust Speaker Representations without Supervision
  5. 2024Self-Distillation Prototypes Network: Learning Robust Speaker Representations without Supervision
  6. 2024Revisiting Self-supervised Learning of Speech Representation from a Mutual Information Perspective
  7. 2023Self-Supervised Learning with Cluster-Aware-DINO for High-Performance Robust Speaker Verification
  8. 2022Label-free Knowledge Distillation with Contrastive Loss for Light-weight Speaker Recognition