mlsanityData to 7 Oct 2026

ESPnet-SPK: full pipeline speaker embedding toolkit with reproducible recipes, self-supervised front-ends, and off-the-shelf models

arXiv 2401.17230 ↗First posted 30 Jan 2024cs.SD cs.AI eess.AScited by 22

Not on the map. Why?

Abstract (arXiv)

This paper introduces ESPnet-SPK, a toolkit designed with several objectives for training speaker embedding extractors. First, we provide an open-source platform for researchers in the speaker recognition community to effortlessly build models. We provide several models, ranging from x-vector to recent SKA-TDNN. Through the modularized architecture design, variants can be developed easily. We also aspire to bridge developed models with other domains, facilitating the broad research community to effortlessly incorporate state-of-the-art embedding extractors. Pre-trained embedding extractors can be accessed in an off-the-shelf manner and we demonstrate the toolkit's versatility by showcasing its integration with two tasks. Another goal is to integrate with diverse self-supervised learning features. We release a reproducible recipe that achieves an equal error rate of 0.39% on the Vox1-O evaluation protocol using WavLM-Large with ECAPA-TDNN.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 46 of its 57 references that are in the map. First posted on arXiv 30 Jan 2024.

Builds on

46 of 57 references.

References · 46

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2015Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shiftcited by 6,979
  3. 2016Layer Normalizationcited by 5,579
  4. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  5. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  6. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  7. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  8. 2018VoxCeleb2: Deep Speaker Recognitioncited by 862
  9. 2017VoxCeleb: a large-scale speaker identification datasetcited by 714
  10. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 652
  11. 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 555
  12. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 554
34 more
  1. 2021SpeechBrain: A General-Purpose Speech Toolkitcited by 404
  2. 2022UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022cited by 381
  3. 2017Generalized End-to-End Loss for Speaker Verificationcited by 290
  4. 2021YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyonecited by 277
  5. 2019NeMo: a toolkit for building AI applications using Neural Modulescited by 230
  6. 2015Deep clustering: Discriminative embeddings for segmentation and separationcited by 211
  7. 2021AASIST: Audio Anti-Spoofing using Integrated Spectro-Temporal Graph Attention Networkscited by 173
  8. 2020In defence of metric learning for speaker recognitioncited by 123
  9. 2021A Review of Speaker Diarization: Recent Advances with Deep Learningcited by 122
  10. 2018Attentive Statistics Pooling for Deep Speaker Embeddingcited by 108
  11. 2022Wespeaker: A Research and Production oriented Speaker Embedding Learning Toolkitcited by 92
  12. 2021UniSpeech: Unified Speech Representation Learning with Labeled and Unlabeled Datacited by 76
  13. 2018VoiceFilter: Targeted Voice Separation by Speaker-Conditioned Spectrogram Maskingcited by 72
  14. 2021Large-scale Self-Supervised Speech Representation Learning for Automatic Speaker Verificationcited by 71
  15. 2022Pushing the limits of raw waveform speaker recognitioncited by 49
  16. 2022MFA-Conformer: Multi-scale Feature Aggregation Conformer for Automatic Speaker Verificationcited by 44
  17. 2019RawNet: Advanced end-to-end deep neural network using raw waveforms for text-independent speaker verificationcited by 33
  18. 2020SpEx: Multi-Scale Time Domain Speaker Extraction Networkcited by 29
  19. 2023The Singing Voice Conversion Challenge 2023cited by 25
  20. 2021ESPnet2-TTS: Extending the Edge of TTS Researchcited by 22
  21. 2020Improving speaker discrimination of target speech extraction with time-domain SpeakerBeamcited by 20
  22. 2021The SpeakIn System for VoxCeleb Speaker Recognition Challange 2021cited by 20
  23. 2022SASV 2022: The First Spoofing-Aware Speaker Verification Challengecited by 19
  24. 20233D-Speaker: A Large-Scale Multi-Device, Multi-Distance, and Multi-Dialect Corpus for Speech Representation Disentanglementcited by 15
  25. 2020How Far Are We from Robust Voice Conversion: A Surveycited by 14
  26. 2022Frequency and Multi-Scale Selective Kernel Attention for Speaker Verificationcited by 10
  27. 2022PaddleSpeech: An Easy-to-Use All-in-One Speech Toolkitcited by 9
  28. 2023VoxBlink: A Large Scale Speaker Verification Dataset on Cameracited by 9
  29. 2022Parameter-efficient transfer learning of pre-trained Transformer models for speaker verification using adapterscited by 8
  30. 2022In search of strong embedding extractors for speaker diarisationcited by 5
  31. 2023The ID R&D VoxCeleb Speaker Recognition Challenge 2023 System Descriptioncited by 4
  32. 2022Quantitative Evidence on Overlooked Aspects of Enrollment Speaker Embeddings for Target Speaker Separationcited by 3
  33. 2022Build a SRE Challenge System: Lessons from VoxSRC 2022 and CNSRC 2022cited by 3
  34. 2022Probing Deep Speaker Embeddings for Speaker-related Taskscited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 7; 2025: 10; 2026: 5.

22 papers.

Sort:

By area:

  1. 2025Recent Advances in Discrete Speech Tokens: A Reviewcited by 42
  2. 2024SpoofCeleb: Speech Deepfake Detection and SASV In The Wildcited by 22
  3. 2024Preference Alignment Improves Language Model-Based TTScited by 21
  4. 2025Nes2Net: A Lightweight Nested Architecture for Foundation Model Driven Speech Anti-spoofingcited by 17
  5. 2024VoiceDiT: Dual-Condition Diffusion Transformer for Environment-Aware Speech Synthesiscited by 10
  6. 2025ESPnet-SpeechLM: An Open Speech Language Model Toolkitcited by 9
  7. 2025OpusLM: A Family of Open Unified Speech Language Modelscited by 9
  8. 2024LSCodec: Low-Bitrate and Speaker-Decoupled Discrete Speech Codeccited by 7
  9. 2025TTSDS2: Resources and Benchmark for Evaluating Human-Quality Text to Speech Systemscited by 7
  10. 2024vec2wav 2.0: Advancing Voice Conversion via Discrete Token Vocoderscited by 6
  11. 2025Benchmarking Training Paradigms, Dataset Composition, and Model Scaling for Child ASR in ESPnetcited by 6
  12. 2024To what extent can ASV systems naturally defend against spoofing attacks?cited by 1
10 more
  1. 2025A Non-autoregressive Model for Joint STT and TTScited by 1
  2. 2025Voice Quality Dimensions as Interpretable Primitives for Speaking Style for Atypical Speech and Affectcited by 1
  3. 2026Do Neural Codecs Generalize? A Controlled Study Across Unseen Languages and Non-Speech Taskscited by 1
  4. 2024ESPnet-EZ: Python-only ESPnet for Easy Fine-tuning and Integrationcited by 0
  5. 2025Geolocation-Aware Robust Spoken Language Identificationcited by 0
  6. 2025Assessing speech quality metrics for evaluation of neural audio codecs under clean speech conditionscited by 0
  7. 2026Probing Low Frame Rate Degradation in Neural Audio Codecscited by 0
  8. 2026Kiwano: A Cutting-Edge Open-Source Toolkit for Speaker Verificationcited by 0
  9. 2026StreamAlign: Streaming Text-Aligned Speech Tokenizationcited by 0
  10. 2026Challenges of Multi-Speaker Extraction for Real Conversational Speech Enhancementcited by 0