ESPnet-SPK: full pipeline speaker embedding toolkit with reproducible recipes, self-supervised front-ends, and off-the-shelf models
Not on the map. Why?
Abstract (arXiv)
This paper introduces ESPnet-SPK, a toolkit designed with several objectives for training speaker embedding extractors. First, we provide an open-source platform for researchers in the speaker recognition community to effortlessly build models. We provide several models, ranging from x-vector to recent SKA-TDNN. Through the modularized architecture design, variants can be developed easily. We also aspire to bridge developed models with other domains, facilitating the broad research community to effortlessly incorporate state-of-the-art embedding extractors. Pre-trained embedding extractors can be accessed in an off-the-shelf manner and we demonstrate the toolkit's versatility by showcasing its integration with two tasks. Another goal is to integrate with diverse self-supervised learning features. We release a reproducible recipe that achieves an equal error rate of 0.39% on the Vox1-O evaluation protocol using WavLM-Large with ECAPA-TDNN.
Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 46 of its 57 references that are in the map. First posted on arXiv 30 Jan 2024.
Builds on
46 of 57 references.
References · 46
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
- 2016Layer Normalizationcited by 5,579
- 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
- 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
- 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
- 2018VoxCeleb2: Deep Speaker Recognitioncited by 862
- 2017VoxCeleb: a large-scale speaker identification datasetcited by 714
- 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 652
- 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 555
- 2018ESPnet: End-to-End Speech Processing Toolkitcited by 554
34 more
- 2021SpeechBrain: A General-Purpose Speech Toolkitcited by 404
- 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 381
- 2017Generalized End-to-End Loss for Speaker Verificationcited by 290
- 2021YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyonecited by 277
- 2019NeMo: a toolkit for building AI applications using Neural Modulescited by 230
- 2015Deep clustering: Discriminative embeddings for segmentation and separationcited by 211
- 2021AASIST: Audio Anti-Spoofing using Integrated Spectro-Temporal Graph Attention Networkscited by 173
- 2020In defence of metric learning for speaker recognitioncited by 123
- 2021A Review of Speaker Diarization: Recent Advances with Deep Learningcited by 122
- 2018Attentive Statistics Pooling for Deep Speaker Embeddingcited by 108
- 2022Wespeaker: A Research and Production oriented Speaker Embedding Learning Toolkitcited by 92
- 2021UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled Datacited by 76
- 2018VoiceFilter: Targeted Voice Separation by Speaker-Conditioned Spectrogram Maskingcited by 72
- 2021Large-scale Self-Supervised Speech Representation Learning for Automatic Speaker Verificationcited by 71
- 2022Pushing the limits of raw waveform speaker recognitioncited by 49
- 2022MFA-Conformer: Multi-scale Feature Aggregation Conformer for Automatic Speaker Verificationcited by 44
- 2019RawNet: Advanced end-to-end deep neural network using raw waveforms for text-independent speaker verificationcited by 33
- 2020SpEx: Multi-Scale Time Domain Speaker Extraction Networkcited by 29
- 2023The Singing Voice Conversion Challenge 2023cited by 25
- 2021ESPnet2-TTS: Extending the Edge of TTS Researchcited by 22
- 2020Improving speaker discrimination of target speech extraction with time-domain SpeakerBeamcited by 20
- 2021The SpeakIn System for VoxCeleb Speaker Recognition Challange 2021cited by 20
- 2022SASV 2022: The First Spoofing-Aware Speaker Verification Challengecited by 19
- 20233D-Speaker: A Large-Scale Multi-Device, Multi-Distance, and Multi-Dialect Corpus for Speech Representation Disentanglementcited by 15
- 2020How Far Are We from Robust Voice Conversion: A Surveycited by 14
- 2022Frequency and Multi-Scale Selective Kernel Attention for Speaker Verificationcited by 10
- 2022PaddleSpeech: An Easy-to-Use All-in-One Speech Toolkitcited by 9
- 2023VoxBlink: A Large Scale Speaker Verification Dataset on Cameracited by 9
- 2022Parameter-efficient transfer learning of pre-trained Transformer models for speaker verification using adapterscited by 8
- 2022In search of strong embedding extractors for speaker diarisationcited by 5
- 2023The ID R&D VoxCeleb Speaker Recognition Challenge 2023 System Descriptioncited by 4
- 2022Quantitative Evidence on Overlooked Aspects of Enrollment Speaker Embeddings for Target Speaker Separationcited by 3
- 2022Build a SRE Challenge System: Lessons from VoxSRC 2022 and CNSRC 2022cited by 3
- 2022Probing Deep Speaker Embeddings for Speaker-related Taskscited by 2
Built on by
Citing papers per year: 2024: 7; 2025: 10; 2026: 5.
22 papers.
Sort:
By area:
- 2025Recent Advances in Discrete Speech Tokens: A Reviewcited by 42
- 2024SpoofCeleb: Speech Deepfake Detection and SASV In The Wildcited by 22
- 2024Preference Alignment Improves Language Model-Based TTScited by 21
- 2025Nes2Net: A Lightweight Nested Architecture for Foundation Model Driven Speech Anti-spoofingcited by 17
- 2024VoiceDiT: Dual-Condition Diffusion Transformer for Environment-Aware Speech Synthesiscited by 10
- 2025ESPnet-SpeechLM: An Open Speech Language Model Toolkitcited by 9
- 2025OpusLM: A Family of Open Unified Speech Language Modelscited by 9
- 2024LSCodec: Low-Bitrate and Speaker-Decoupled Discrete Speech Codeccited by 7
- 2025TTSDS2: Resources and Benchmark for Evaluating Human-Quality Text to Speech Systemscited by 7
- 2024vec2wav 2.0: Advancing Voice Conversion via Discrete Token Vocoderscited by 6
- 2025Benchmarking Training Paradigms, Dataset Composition, and Model Scaling for Child ASR in ESPnetcited by 6
- 2024To what extent can ASV systems naturally defend against spoofing attacks?cited by 1
10 more
- 2025A Non-autoregressive Model for Joint STT and TTScited by 1
- 2025Voice Quality Dimensions as Interpretable Primitives for Speaking Style for Atypical Speech and Affectcited by 1
- 2026Do Neural Codecs Generalize? A Controlled Study Across Unseen Languages and Non-Speech Taskscited by 1
- 2024ESPnet-EZ: Python-only ESPnet for Easy Fine-tuning and Integrationcited by 0
- 2025Geolocation-Aware Robust Spoken Language Identificationcited by 0
- 2025Assessing speech quality metrics for evaluation of neural audio codecs under clean speech conditionscited by 0
- 2026Probing Low Frame Rate Degradation in Neural Audio Codecscited by 0
- 2026Kiwano: A Cutting-Edge Open-Source Toolkit for Speaker Verificationcited by 0
- 2026StreamAlign: Streaming Text-Aligned Speech Tokenizationcited by 0
- 2026Challenges of Multi-Speaker Extraction for Real Conversational Speech Enhancementcited by 0
- 2026Challenges of Multi-Speaker Extraction for Real Conversational Speech Enhancement22 Sep 2026
- 2026StreamAlign: Streaming Text-Aligned Speech Tokenization9 Sep 2026
- 2026Kiwano: A Cutting-Edge Open-Source Toolkit for Speaker Verification21 Jun 2026
- 2026Probing Low Frame Rate Degradation in Neural Audio Codecs15 Jun 2026
- 2026Do Neural Codecs Generalize? A Controlled Study Across Unseen Languages and Non-Speech Tasks18 Jan 2026
- 2025Assessing speech quality metrics for evaluation of neural audio codecs under clean speech conditions29 Sep 2025
- 2025Geolocation-Aware Robust Spoken Language Identification23 Aug 2025
- 2025Benchmarking Training Paradigms, Dataset Composition, and Model Scaling for Child ASR in ESPnet22 Aug 2025
- 2025TTSDS2: Resources and Benchmark for Evaluating Human-Quality Text to Speech Systems24 Jun 2025
- 2025OpusLM: A Family of Open Unified Speech Language Models21 Jun 2025
- 2025Voice Quality Dimensions as Interpretable Primitives for Speaking Style for Atypical Speech and Affect27 May 2025
- 2025Nes2Net: A Lightweight Nested Architecture for Foundation Model Driven Speech Anti-spoofing8 Apr 2025
10 more
- 2025ESPnet-SpeechLM: An Open Speech Language Model Toolkit21 Feb 2025
- 2025Recent Advances in Discrete Speech Tokens: A Review10 Feb 2025
- 2025A Non-autoregressive Model for Joint STT and TTS15 Jan 2025
- 2024VoiceDiT: Dual-Condition Diffusion Transformer for Environment-Aware Speech Synthesis26 Dec 2024
- 2024LSCodec: Low-Bitrate and Speaker-Decoupled Discrete Speech Codec21 Oct 2024
- 2024Preference Alignment Improves Language Model-Based TTS19 Sep 2024
- 2024SpoofCeleb: Speech Deepfake Detection and SASV In The Wild18 Sep 2024
- 2024ESPnet-EZ: Python-only ESPnet for Easy Fine-tuning and Integration14 Sep 2024
- 2024vec2wav 2.0: Advancing Voice Conversion via Discrete Token Vocoders3 Sep 2024
- 2024To what extent can ASV systems naturally defend against spoofing attacks?8 Jun 2024
- 2025ESPnet-SpeechLM: An Open Speech Language Model Toolkitcited by 9
- 2025ESPnet-SpeechLM: An Open Speech Language Model Toolkit21 Feb 2025
- 2024Preference Alignment Improves Language Model-Based TTScited by 21
- 2024Preference Alignment Improves Language Model-Based TTS19 Sep 2024
- 2025Recent Advances in Discrete Speech Tokens: A Reviewcited by 42
- 2024SpoofCeleb: Speech Deepfake Detection and SASV In The Wildcited by 22
- 2025Nes2Net: A Lightweight Nested Architecture for Foundation Model Driven Speech Anti-spoofingcited by 17
- 2025OpusLM: A Family of Open Unified Speech Language Modelscited by 9
- 2024LSCodec: Low-Bitrate and Speaker-Decoupled Discrete Speech Codeccited by 7
- 2025TTSDS2: Resources and Benchmark for Evaluating Human-Quality Text to Speech Systemscited by 7
- 2024vec2wav 2.0: Advancing Voice Conversion via Discrete Token Vocoderscited by 6
- 2025Benchmarking Training Paradigms, Dataset Composition, and Model Scaling for Child ASR in ESPnetcited by 6
- 2024To what extent can ASV systems naturally defend against spoofing attacks?cited by 1
- 2025A Non-autoregressive Model for Joint STT and TTScited by 1
- 2025Voice Quality Dimensions as Interpretable Primitives for Speaking Style for Atypical Speech and Affectcited by 1
- 2026Do Neural Codecs Generalize? A Controlled Study Across Unseen Languages and Non-Speech Taskscited by 1
12 most cited of 19 All 19 (JSON)
- 2026Challenges of Multi-Speaker Extraction for Real Conversational Speech Enhancement22 Sep 2026
- 2026StreamAlign: Streaming Text-Aligned Speech Tokenization9 Sep 2026
- 2026Kiwano: A Cutting-Edge Open-Source Toolkit for Speaker Verification21 Jun 2026
- 2026Probing Low Frame Rate Degradation in Neural Audio Codecs15 Jun 2026
- 2026Do Neural Codecs Generalize? A Controlled Study Across Unseen Languages and Non-Speech Tasks18 Jan 2026
- 2025Assessing speech quality metrics for evaluation of neural audio codecs under clean speech conditions29 Sep 2025
- 2025Geolocation-Aware Robust Spoken Language Identification23 Aug 2025
- 2025Benchmarking Training Paradigms, Dataset Composition, and Model Scaling for Child ASR in ESPnet22 Aug 2025
- 2025TTSDS2: Resources and Benchmark for Evaluating Human-Quality Text to Speech Systems24 Jun 2025
- 2025OpusLM: A Family of Open Unified Speech Language Models21 Jun 2025
- 2025Voice Quality Dimensions as Interpretable Primitives for Speaking Style for Atypical Speech and Affect27 May 2025
- 2025Nes2Net: A Lightweight Nested Architecture for Foundation Model Driven Speech Anti-spoofing8 Apr 2025
12 newest of 19 All 19 (JSON)