# Pushing the limits of raw waveform speaker recognition

arXiv: https://arxiv.org/abs/2203.08488 · HTML: https://mlsanity.com/paper/2203.08488/

## Key facts

- Cited by 51 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 48 references that are in the map.
- First posted on arXiv 16 Mar 2022.

## Builds on (most cited first, 30 of 34)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [Bootstrap your own latent: A new approach to self-supervised Learning](https://mlsanity.com/paper/2006.07733/) (2020)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [BEiT: BERT Pre-Training of Image Transformers](https://mlsanity.com/paper/2106.08254/) (2021)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Instance Normalization: The Missing Ingredient for Fast Stylization](https://mlsanity.com/paper/1607.08022/) (2016)
- [VoxCeleb2: Deep Speaker Recognition](https://mlsanity.com/paper/1806.05622/) (2018)
- [VoxCeleb: a large-scale speaker identification dataset](https://mlsanity.com/paper/1706.08612/) (2017)
- [data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language](https://mlsanity.com/paper/2202.03555/) (2022)
- [ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification](https://mlsanity.com/paper/2005.07143/) (2020)
- [Res2Net: A New Multi-scale Backbone Architecture](https://mlsanity.com/paper/1904.01169/) (2019)
- [SpeechBrain: A General-Purpose Speech Toolkit](https://mlsanity.com/paper/2106.04624/) (2021)
- [S4L: Self-Supervised Semi-Supervised Learning](https://mlsanity.com/paper/1905.03670/) (2019)
- [Speaker Recognition from Raw Waveform with SincNet](https://mlsanity.com/paper/1808.00158/) (2018)
- [RawNet: Advanced end-to-end deep neural network using raw waveforms for text-independent speaker verification](https://mlsanity.com/paper/1904.08104/) (2019)
- [Self-supervised Text-independent Speaker Verification using Prototypical Momentum Contrastive Learning](https://mlsanity.com/paper/2012.07178/) (2020)
- [Augmentation adversarial training for self-supervised speaker recognition](https://mlsanity.com/paper/2007.12085/) (2020)
- [Self-Supervised Speaker Verification with Simple Siamese Network and Self-Supervised Regularization](https://mlsanity.com/paper/2112.04459/) (2021)
- [Filterbank design for end-to-end speech separation](https://mlsanity.com/paper/1910.10400/) (2019)
- [Self-supervised Speaker Recognition with Loss-gated Learning](https://mlsanity.com/paper/2110.03869/) (2021)
- [Improved RawNet with Feature Map Scaling for Text-independent Speaker Verification using Raw Waveforms](https://mlsanity.com/paper/2004.00526/) (2020)
- [An iterative framework for self-supervised deep speaker representation learning](https://mlsanity.com/paper/2010.14751/) (2020)
- [RawNeXt: Speaker verification system for variable-duration utterances with deep layer aggregation and extended dynamic scaling policies](https://mlsanity.com/paper/2112.07935/) (2021)

## Built on by (most cited first, 30 of 51)

- [Progressive Disentangled Representation Learning for Fine-Grained Controllable Talking Head Synthesis](https://mlsanity.com/paper/2211.14506/) (2022)
- [Improved DeepFake Detection Using Whisper Features](https://mlsanity.com/paper/2306.01428/) (2023)
- [The Singing Voice Conversion Challenge 2023](https://mlsanity.com/paper/2306.14422/) (2023)
- [ESPnet-SPK: full pipeline speaker embedding toolkit with reproducible recipes, self-supervised front-ends, and off-the-shelf models](https://mlsanity.com/paper/2401.17230/) (2024)
- [SpoofCeleb: Speech Deepfake Detection and SASV In The Wild](https://mlsanity.com/paper/2409.17285/) (2024)
- [DiffProsody: Diffusion-based Latent Prosody Generation for Expressive Speech Synthesis with Prosody Conditional Adversarial Training](https://mlsanity.com/paper/2307.16549/) (2023)
- [SLMIA-SR: Speaker-Level Membership Inference Attacks against Speaker Recognition Systems](https://mlsanity.com/paper/2309.07983/) (2023)
- [I Can Hear You: Selective Robust Training for Deepfake Audio Detection](https://mlsanity.com/paper/2411.00121/) (2024)
- [Defense Against Adversarial Attacks on Audio DeepFake Detection](https://mlsanity.com/paper/2212.14597/) (2022)
- [Frequency and Multi-Scale Selective Kernel Attention for Speaker Verification](https://mlsanity.com/paper/2204.01005/) (2022)
- [Fake the Real: Backdoor Attack on Deep Speech Classification via Voice Conversion](https://mlsanity.com/paper/2306.15875/) (2023)
- [AASIST3: KAN-Enhanced AASIST Speech Deepfake Detection using SSL Features and Additional Regularization for the ASVspoof 2024 Challenge](https://mlsanity.com/paper/2408.17352/) (2024)
- [Adversarial Attacks on Audio Deepfake Detection: A Benchmark and Comparative Study](https://mlsanity.com/paper/2509.07132/) (2025)
- [Singer Identity Representation Learning using Self-Supervised Techniques](https://mlsanity.com/paper/2401.05064/) (2024)
- [a-DCF: an architecture agnostic metric with application to spoofing-robust speaker verification](https://mlsanity.com/paper/2403.01355/) (2024)
- [Benchmarking Training Paradigms, Dataset Composition, and Model Scaling for Child ASR in ESPnet](https://mlsanity.com/paper/2508.16576/) (2025)
- [In search of strong embedding extractors for speaker diarisation](https://mlsanity.com/paper/2210.14682/) (2022)
- [PASE: Leveraging the Phonological Prior of WavLM for Low-Hallucination Generative Speech Enhancement](https://mlsanity.com/paper/2511.13300/) (2025)
- [Large-scale learning of generalised representations for speaker recognition](https://mlsanity.com/paper/2210.10985/) (2022)
- [PITS: Variational Pitch Inference without Fundamental Frequency for End-to-End Pitch-controllable TTS](https://mlsanity.com/paper/2302.12391/) (2023)
- [Summary of the DISPLACE Challenge 2023 - DIarization of SPeaker and LAnguage in Conversational Environments](https://mlsanity.com/paper/2311.12564/) (2023)
- [CrossSpeech: Speaker-independent Acoustic Representation for Cross-lingual Speech Synthesis](https://mlsanity.com/paper/2302.14370/) (2023)
- [Breaking Speaker Recognition with PaddingBack](https://mlsanity.com/paper/2308.04179/) (2023)
- [DeepFake-O-Meter v2.0: An Open Platform for DeepFake Detection](https://mlsanity.com/paper/2404.13146/) (2024)
- [IDFace: Face Template Protection for Efficient and Secure Identification](https://mlsanity.com/paper/2507.12050/) (2025)
- [SHIELD: A Secure and Highly Enhanced Integrated Learning for Robust Deepfake Detection against Adversarial Attacks](https://mlsanity.com/paper/2507.13170/) (2025)
- [VSSFlow: Unifying Video-conditioned Sound and Speech Generation via Joint Learning](https://mlsanity.com/paper/2509.24773/) (2025)
- [A Study on Bias and Fairness In Deep Speaker Recognition](https://mlsanity.com/paper/2303.08026/) (2023)
- [Scores Know Bobs Voice: Speaker Impersonation Attack](https://mlsanity.com/paper/2603.02781/) (2026)
- [The ReturnZero System for VoxCeleb Speaker Recognition Challenge 2022](https://mlsanity.com/paper/2209.10147/) (2022)

## Abstract (arXiv)

In recent years, speaker recognition systems based on raw waveform inputs have received increasing attention. However, the performance of such systems are typically inferior to the state-of-the-art handcrafted feature-based counterparts, which demonstrate equal error rates under 1% on the popular VoxCeleb1 test set. This paper proposes a novel speaker recognition model based on raw waveform inputs. The model incorporates recent advances in machine learning and speaker verification, including the Res2Net backbone module and multi-layer feature aggregation. Our best model achieves an equal error rate of 0.89%, which is competitive with the state-of-the-art models based on handcrafted features, and outperforms the best model based on raw waveform inputs by a large margin. We also explore the application of the proposed model in the context of self-supervised learning framework. Our self-supervised model outperforms single phase-based existing works in this line of research. Finally, we show that self-supervised pre-training is effective for the semi-supervised scenario where we only have a small set of labelled training data, along with a larger set of unlabelled examples.