# Large-scale Self-Supervised Speech Representation Learning for Automatic Speaker Verification

arXiv: https://arxiv.org/abs/2110.05777 · HTML: https://mlsanity.com/paper/2110.05777/

## Key facts

- Cited by 78 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Oct 2021.

## Built on by (most cited first, 30 of 78)

- [F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching](https://mlsanity.com/paper/2410.06885/) (2024)
- [CosyVoice 3: Towards In-the-wild Speech Generation via Scaling-up and Post-training](https://mlsanity.com/paper/2505.17589/) (2025)
- [Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokens](https://mlsanity.com/paper/2503.01710/) (2025)
- [FreeVC: Towards High-Quality Text-Free One-Shot Voice Conversion](https://mlsanity.com/paper/2210.15418/) (2022)
- [VALL-E R: Robust and Efficient Zero-Shot Text-to-Speech Synthesis via Monotonic Alignment](https://mlsanity.com/paper/2406.07855/) (2024)
- [MovieBench: A Hierarchical Movie Level Dataset for Long Video Generation](https://mlsanity.com/paper/2411.15262/) (2024)
- [Why does Self-Supervised Learning for Speech Recognition Benefit Speaker Recognition?](https://mlsanity.com/paper/2204.12765/) (2022)
- [SplitMeanFlow: Interval Splitting Consistency in Few-Step Generative Modeling](https://mlsanity.com/paper/2507.16884/) (2025)
- [F5R-TTS: Improving Flow-Matching based Text-to-Speech with Group Relative Policy Optimization](https://mlsanity.com/paper/2504.02407/) (2025)
- [ESPnet-SPK: full pipeline speaker embedding toolkit with reproducible recipes, self-supervised front-ends, and off-the-shelf models](https://mlsanity.com/paper/2401.17230/) (2024)
- [LeBenchmark 2.0: a Standardized, Replicable and Enhanced Framework for Self-supervised Representations of French Speech](https://mlsanity.com/paper/2309.05472/) (2023)
- [Supervision-Guided Codebooks for Masked Prediction in Speech Pre-training](https://mlsanity.com/paper/2206.10125/) (2022)
- [Leveraging Speech PTM, Text LLM, and Emotional TTS for Speech Emotion Recognition](https://mlsanity.com/paper/2309.10294/) (2023)
- [Phoneme Segmentation Using Self-Supervised Speech Models](https://mlsanity.com/paper/2211.01461/) (2022)
- [Adapting Multi-Lingual ASR Models for Handling Multiple Talkers](https://mlsanity.com/paper/2305.18747/) (2023)
- [Fast-HuBERT: An Efficient Training Framework for Self-Supervised Speech Representation Learning](https://mlsanity.com/paper/2309.13860/) (2023)
- [DurFlex-EVC: Duration-Flexible Emotional Voice Conversion Leveraging Discrete Representations without Text Alignment](https://mlsanity.com/paper/2401.08095/) (2024)
- [On the Use of Semantically-Aligned Speech Representations for Spoken Language Understanding](https://mlsanity.com/paper/2210.05291/) (2022)
- [Towards single integrated spoofing-aware speaker verification embeddings](https://mlsanity.com/paper/2305.19051/) (2023)
- [Leveraging ASR Pretrained Conformers for Speaker Verification through Transfer Learning and Knowledge Distillation](https://mlsanity.com/paper/2309.03019/) (2023)
- [Speech Emotion: Investigating Model Representations, Multi-Task Learning and Knowledge Distillation](https://mlsanity.com/paper/2207.03334/) (2022)
- [Speech separation with large-scale self-supervised learning](https://mlsanity.com/paper/2211.05172/) (2022)
- [DeepAudio-V1:Towards Multi-Modal Multi-Stage End-to-End Video to Speech and Audio Generation](https://mlsanity.com/paper/2503.22265/) (2025)
- [FlowTTS-GRPO: Online Reinforcement Learning with Multi-Objective Reward Optimization for Flow-Matching Based Text-to-Speech](https://mlsanity.com/paper/2606.23190/) (2026)
- [Large-scale learning of generalised representations for speaker recognition](https://mlsanity.com/paper/2210.10985/) (2022)
- [Transferring speech-generic and depression-specific knowledge for Alzheimer's disease detection](https://mlsanity.com/paper/2310.04358/) (2023)
- [HiddenSpeaker: Generate Imperceptible Unlearnable Audios for Speaker Verification System](https://mlsanity.com/paper/2405.15655/) (2024)
- [TS-SUPERB: A Target Speech Processing Benchmark for Speech Self-Supervised Learning Models](https://mlsanity.com/paper/2505.06660/) (2025)
- [Learning the joint distribution of two sequences using little or no paired data](https://mlsanity.com/paper/2212.03232/) (2022)
- [One-Step Knowledge Distillation and Fine-Tuning in Using Large Pre-Trained Self-Supervised Learning Models for Speaker Verification](https://mlsanity.com/paper/2305.17394/) (2023)

## Abstract (arXiv)

The speech representations learned from large-scale unlabeled data have shown better generalizability than those from supervised learning and thus attract a lot of interest to be applied for various downstream tasks. In this paper, we explore the limits of speech representations learned by different self-supervised objectives and datasets for automatic speaker verification (ASV), especially with a well-recognized SOTA ASV model, ECAPA-TDNN [1], as a downstream model. The representations from all hidden layers of the pre-trained model are firstly averaged with learnable weights and then fed into the ECAPA-TDNN as input features. The experimental results on Voxceleb dataset show that the weighted average representation is significantly superior to FBank, a conventional handcrafted feature for ASV. Our best single system achieves 0.537%, 0.569%, and 1.180% equal error rate (EER) on the three official trials of VoxCeleb1, separately. Accordingly, the ensemble system with three pre-trained models can further improve the EER to 0.479%, 0.536% and 1.023%. Among the three evaluation trials, our best system outperforms the winner system [2] of the VoxCeleb Speaker Recognition Challenge 2021 (VoxSRC2021) on the VoxCeleb1-E trial.