mlsanityData to 14 Oct 2026

Large-scale Self-Supervised Speech Representation Learning for Automatic Speaker Verification

arXiv 2110.05777 ↗First posted 12 Oct 2021cs.SD eess.AScited by 78

Not on the map. Why?

Abstract (arXiv)

The speech representations learned from large-scale unlabeled data have shown better generalizability than those from supervised learning and thus attract a lot of interest to be applied for various downstream tasks. In this paper, we explore the limits of speech representations learned by different self-supervised objectives and datasets for automatic speaker verification (ASV), especially with a well-recognized SOTA ASV model, ECAPA-TDNN [1], as a downstream model. The representations from all hidden layers of the pre-trained model are firstly averaged with learnable weights and then fed into the ECAPA-TDNN as input features. The experimental results on Voxceleb dataset show that the weighted average representation is significantly superior to FBank, a conventional handcrafted feature for ASV. Our best single system achieves 0.537%, 0.569%, and 1.180% equal error rate (EER) on the three official trials of VoxCeleb1, separately. Accordingly, the ensemble system with three pre-trained models can further improve the EER to 0.479%, 0.536% and 1.023%. Among the three evaluation trials, our best system outperforms the winner system [2] of the VoxCeleb Speaker Recognition Challenge 2021 (VoxSRC2021) on the VoxCeleb1-E trial.

Key facts. Cited by 78 papers in the map (data through 14 Oct 2026). First posted on arXiv 12 Oct 2021.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 13; 2023: 14; 2024: 12; 2025: 19; 2026: 20.

78 papers.

Sort:

By area:

  1. 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matchingcited by 361
  2. 2025CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-trainingcited by 150
  3. 2025Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokenscited by 113
  4. 2022FreeVC: Towards High-Quality Text-Free One-Shot Voice Conversioncited by 80
  5. 2024VALL-E R: Robust and Efficient Zero-Shot Text-to-Speech Synthesis via Monotonic Alignmentcited by 38
  6. 2024MovieBench: A Hierarchical Movie Level Dataset for Long Video Generationcited by 36
  7. 2022Why does Self-Supervised Learning for Speech Recognition Benefit Speaker Recognition?cited by 27
  8. 2025SplitMeanFlow: Interval Splitting Consistency in Few-Step Generative Modelingcited by 26
  9. 2025F5R-TTS: Improving Flow-Matching based Text-to-Speech with Group Relative Policy Optimizationcited by 24
  10. 2024ESPnet-SPK: full pipeline speaker embedding toolkit with reproducible recipes, self-supervised front-ends, and off-the-shelf modelscited by 22
  11. 2023LeBenchmark 2.0: a Standardized, Replicable and Enhanced Framework for Self-supervised Representations of French Speechcited by 18
  12. 2022Supervision-Guided Codebooks for Masked Prediction in Speech Pre-trainingcited by 14
38 more
  1. 2023Leveraging Speech PTM, Text LLM, and Emotional TTS for Speech Emotion Recognitioncited by 12
  2. 2022Phoneme Segmentation Using Self-Supervised Speech Modelscited by 11
  3. 2023Adapting Multi-Lingual ASR Models for Handling Multiple Talkerscited by 11
  4. 2023Fast-HuBERT: An Efficient Training Framework for Self-Supervised Speech Representation Learningcited by 9
  5. 2024DurFlex-EVC: Duration-Flexible Emotional Voice Conversion Leveraging Discrete Representations without Text Alignmentcited by 8
  6. 2022On the Use of Semantically-Aligned Speech Representations for Spoken Language Understandingcited by 7
  7. 2023Towards single integrated spoofing-aware speaker verification embeddingscited by 7
  8. 2023Leveraging ASR Pretrained Conformers for Speaker Verification through Transfer Learning and Knowledge Distillationcited by 6
  9. 2022Speech Emotion: Investigating Model Representations, Multi-Task Learning and Knowledge Distillationcited by 5
  10. 2022Speech separation with large-scale self-supervised learningcited by 5
  11. 2025DeepAudio-V1:Towards Multi-Modal Multi-Stage End-to-End Video to Speech and Audio Generationcited by 5
  12. 2026FlowTTS-GRPO: Online Reinforcement Learning with Multi-Objective Reward Optimization for Flow-Matching Based Text-to-Speechcited by 5
  13. 2022Large-scale learning of generalised representations for speaker recognitioncited by 4
  14. 2023Transferring speech-generic and depression-specific knowledge for Alzheimer's disease detectioncited by 4
  15. 2024HiddenSpeaker: Generate Imperceptible Unlearnable Audios for Speaker Verification Systemcited by 4
  16. 2025TS-SUPERB: A Target Speech Processing Benchmark for Speech Self-Supervised Learning Modelscited by 4
  17. 2022Learning the joint distribution of two sequences using little or no paired datacited by 3
  18. 2023One-Step Knowledge Distillation and Fine-Tuning in Using Large Pre-Trained Self-Supervised Learning Models for Speaker Verificationcited by 3
  19. 2025Efficient Finetuning for Dimensional Speech Emotion Recognition in the Age of Transformerscited by 3
  20. 2025Word stress in self-supervised speech models: A cross-linguistic comparisoncited by 3
  21. 2025SenSE: Semantic-Aware High-Fidelity Universal Speech Enhancementcited by 3
  22. 2025Enhancing Speaker Verification with w2v-BERT 2.0 and Knowledge Distillation guided Structured Pruningcited by 3
  23. 2026FlashLabs Chroma 1.0: A Real-Time End-to-End Spoken Dialogue Model with Personalized Voice Cloningcited by 3
  24. 2026Habibi: Laying the Open-Source Foundation of Unified-Dialectal Arabic Speech Synthesiscited by 3
  25. 2026WavCube: Unifying Speech Representation for Understanding and Generation via Semantic-Acoustic Joint Modelingcited by 3
  26. 2023Semantic enrichment towards efficient speech representationscited by 2
  27. 2023Spatial HuBERT: Self-supervised Spatial Speech Representation Learning for a Single Talker from Multi-channel Audiocited by 2
  28. 2024MulliVC: Multi-lingual Voice Conversion With Cycle Consistencycited by 2
  29. 2024Layer-aware TDNN: Speaker Recognition Using Multi-Layer Features from Pre-Trained Modelscited by 2
  30. 2025Towards Flow-Matching-based TTS without Classifier-Free Guidancecited by 2
  31. 2022RGMIM: Region-Guided Masked Image Modeling for Learning Meaningful Representations from X-Ray Imagescited by 1
  32. 2022Label-free Knowledge Distillation with Contrastive Loss for Light-weight Speaker Recognitioncited by 1
  33. 2023Toward Leveraging Pre-Trained Self-Supervised Frontends for Automatic Singing Voice Understanding Tasks: Three Case Studiescited by 1
  34. 2023An Effective Transformer-based Contextual Model and Temporal Gate Pooling for Speaker Identificationcited by 1
  35. 2023QS-TTS: Towards Semi-Supervised Text-to-Speech Synthesis via Vector-Quantized Self-Supervised Speech Representation Learningcited by 1
  36. 2024Self-supervised Reflective Learning through Self-distillation and Online Clustering for Speaker Representation Learningcited by 1
  37. 2024Probing the Information Encoded in Neural-based Acoustic Models of Automatic Speech Recognition Systemscited by 1
  38. 2024SVSNet+: Enhancing Speaker Voice Similarity Assessment Models with Representations from Speech Foundation Modelscited by 1

50 most cited of 78 All 78 (JSON)