# Wespeaker: A Research and Production oriented Speaker Embedding Learning Toolkit

arXiv: https://arxiv.org/abs/2210.17016 · HTML: https://mlsanity.com/paper/2210.17016/

## Key facts

- Cited by 97 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Oct 2022.

## Built on by (most cited first, 30 of 97)

- [Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokens](https://mlsanity.com/paper/2503.01710/) (2025)
- [Autoregressive Diffusion Transformer for Text-to-Speech Synthesis](https://mlsanity.com/paper/2406.05551/) (2024)
- [RepCodec: A Speech Representation Codec for Speech Tokenization](https://mlsanity.com/paper/2309.00169/) (2023)
- [MaskSR: Masked Language Model for Full-band Speech Restoration](https://mlsanity.com/paper/2406.02092/) (2024)
- [F5R-TTS: Improving Flow-Matching based Text-to-Speech with Group Relative Policy Optimization](https://mlsanity.com/paper/2504.02407/) (2025)
- [ESPnet-SPK: full pipeline speaker embedding toolkit with reproducible recipes, self-supervised front-ends, and off-the-shelf models](https://mlsanity.com/paper/2401.17230/) (2024)
- [LLaSE-G1: Incentivizing Generalization Capability for LLaMA-based Speech Enhancement](https://mlsanity.com/paper/2503.00493/) (2025)
- [Coding Speech through Vocal Tract Kinematics](https://mlsanity.com/paper/2406.12998/) (2024)
- [FlowSE: Efficient and High-Quality Speech Enhancement via Flow Matching](https://mlsanity.com/paper/2505.19476/) (2025)
- [TTSDS -- Text-to-Speech Distribution Score](https://mlsanity.com/paper/2407.12707/) (2024)
- [Disentangling Voice and Content with Self-Supervision for Speaker Recognition](https://mlsanity.com/paper/2310.01128/) (2023)
- [What Do Self-Supervised Speech and Speaker Models Learn? New Findings From a Cross Model Layer-Wise Analysis](https://mlsanity.com/paper/2401.17632/) (2024)
- [TSELM: Target Speaker Extraction using Discrete Tokens and Language Models](https://mlsanity.com/paper/2409.07841/) (2024)
- [TouchTTS: An Embarrassingly Simple TTS Framework that Everyone Can Touch](https://mlsanity.com/paper/2412.08237/) (2024)
- [Voila: Voice-Language Foundation Models for Real-Time Autonomous Interaction and Voice Role-Play](https://mlsanity.com/paper/2505.02707/) (2025)
- [Predicting Turn-Taking and Backchannel in Human-Machine Conversations Using Linguistic, Acoustic, and Visual Signals](https://mlsanity.com/paper/2505.12654/) (2025)
- [DDTSE: Discriminative Diffusion Model for Target Speech Extraction](https://mlsanity.com/paper/2309.13874/) (2023)
- [TTSDS2: Resources and Benchmark for Evaluating Human-Quality Text to Speech Systems](https://mlsanity.com/paper/2506.19441/) (2025)
- [DMOSpeech 2: Reinforcement Learning for Duration Prediction in Metric-Optimized Speech Synthesis](https://mlsanity.com/paper/2507.14988/) (2025)
- [A Benchmark for Multi-speaker Anonymization](https://mlsanity.com/paper/2407.05608/) (2024)
- [StyleTTS-ZS: Efficient High-Quality Zero-Shot Text-to-Speech Synthesis with Distilled Time-Varying Style Diffusion](https://mlsanity.com/paper/2409.10058/) (2024)
- [Certification of Speaker Recognition Models to Additive Perturbations](https://mlsanity.com/paper/2404.18791/) (2024)
- [The Second DISPLACE Challenge : DIarization of SPeaker and LAnguage in Conversational Environments](https://mlsanity.com/paper/2406.09494/) (2024)
- [MeanFlowSE: one-step generative speech enhancement via conditional mean flow](https://mlsanity.com/paper/2509.14858/) (2025)
- [UniFlow-Audio: Unified Flow Matching for Audio Generation from Omni-Modalities](https://mlsanity.com/paper/2509.24391/) (2025)
- [A Benchmark and Multi-Agent System for Instruction-driven Cinematic Video Compilation](https://mlsanity.com/paper/2604.10456/) (2026)
- [Joint Semantic Knowledge Distillation and Masked Acoustic Modeling for Full-band Speech Restoration with Improved Intelligibility](https://mlsanity.com/paper/2409.09357/) (2024)
- [DMOSpeech: Direct Metric Optimization via Distilled Diffusion Model in Zero-Shot Speech Synthesis](https://mlsanity.com/paper/2410.11097/) (2024)
- [LauraTSE: Target Speaker Extraction using Auto-Regressive Decoder-Only Language Models](https://mlsanity.com/paper/2504.07402/) (2025)
- [Emphasized Non-Target Speaker Knowledge in Knowledge Distillation for Automatic Speaker Verification](https://mlsanity.com/paper/2309.14838/) (2023)

## Abstract (arXiv)

Speaker modeling is essential for many related tasks, such as speaker recognition and speaker diarization. The dominant modeling approach is fixed-dimensional vector representation, i.e., speaker embedding. This paper introduces a research and production oriented speaker embedding learning toolkit, Wespeaker. Wespeaker contains the implementation of scalable data management, state-of-the-art speaker embedding models, loss functions, and scoring back-ends, with highly competitive results achieved by structured recipes which were adopted in the winning systems in several speaker verification challenges. The application to other downstream tasks such as speaker diarization is also exhibited in the related recipe. Moreover, CPU- and GPU-compatible deployment codes are integrated for production-oriented development. The toolkit is publicly available at https://github.com/wenet-e2e/wespeaker.