Wespeaker: A Research and Production oriented Speaker Embedding Learning Toolkit
Not on the map. Why?
Abstract (arXiv)
Speaker modeling is essential for many related tasks, such as speaker recognition and speaker diarization. The dominant modeling approach is fixed-dimensional vector representation, i.e., speaker embedding. This paper introduces a research and production oriented speaker embedding learning toolkit, Wespeaker. Wespeaker contains the implementation of scalable data management, state-of-the-art speaker embedding models, loss functions, and scoring back-ends, with highly competitive results achieved by structured recipes which were adopted in the winning systems in several speaker verification challenges. The application to other downstream tasks such as speaker diarization is also exhibited in the related recipe. Moreover, CPU- and GPU-compatible deployment codes are integrated for production-oriented development. The toolkit is publicly available at https://github.com/wenet-e2e/wespeaker.
Key facts. Cited by 97 papers in the map (data through 14 Oct 2026). First posted on arXiv 31 Oct 2022.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2023: 7; 2024: 24; 2025: 32; 2026: 34.
97 papers.
Sort:
By area:
- 2025Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokenscited by 113
- 2024Autoregressive Diffusion Transformer for Text-to-Speech Synthesiscited by 34
- 2023RepCodec: A Speech Representation Codec for Speech Tokenizationcited by 27
- 2024MaskSR: Masked Language Model for Full-band Speech Restorationcited by 24
- 2025F5R-TTS: Improving Flow-Matching based Text-to-Speech with Group Relative Policy Optimizationcited by 24
- 2024ESPnet-SPK: full pipeline speaker embedding toolkit with reproducible recipes, self-supervised front-ends, and off-the-shelf modelscited by 22
- 2025LLaSE-G1: Incentivizing Generalization Capability for LLaMA-based Speech Enhancementcited by 19
- 2024Coding Speech through Vocal Tract Kinematicscited by 18
- 2025FlowSE: Efficient and High-Quality Speech Enhancement via Flow Matchingcited by 17
- 2024TTSDS -- Text-to-Speech Distribution Scorecited by 14
- 2023Disentangling Voice and Content with Self-Supervision for Speaker Recognitioncited by 13
- 2024What Do Self-Supervised Speech and Speaker Models Learn? New Findings From a Cross Model Layer-Wise Analysiscited by 11
38 more
- 2024TSELM: Target Speaker Extraction using Discrete Tokens and Language Modelscited by 10
- 2024TouchTTS: An Embarrassingly Simple TTS Framework that Everyone Can Touchcited by 10
- 2025Voila: Voice-Language Foundation Models for Real-Time Autonomous Interaction and Voice Role-Playcited by 10
- 2025Predicting Turn-Taking and Backchannel in Human-Machine Conversations Using Linguistic, Acoustic, and Visual Signalscited by 10
- 2023DDTSE: Discriminative Diffusion Model for Target Speech Extractioncited by 8
- 2025TTSDS2: Resources and Benchmark for Evaluating Human-Quality Text to Speech Systemscited by 8
- 2025DMOSpeech 2: Reinforcement Learning for Duration Prediction in Metric-Optimized Speech Synthesiscited by 8
- 2024A Benchmark for Multi-speaker Anonymizationcited by 7
- 2024StyleTTS-ZS: Efficient High-Quality Zero-Shot Text-to-Speech Synthesis with Distilled Time-Varying Style Diffusioncited by 7
- 2024Certification of Speaker Recognition Models to Additive Perturbationscited by 6
- 2024The Second DISPLACE Challenge : DIarization of SPeaker and LAnguage in Conversational Environmentscited by 6
- 2025MeanFlowSE: one-step generative speech enhancement via conditional mean flowcited by 6
- 2025UniFlow-Audio: Unified Flow Matching for Audio Generation from Omni-Modalitiescited by 6
- 2026A Benchmark and Multi-Agent System for Instruction-driven Cinematic Video Compilationcited by 6
- 2024Joint Semantic Knowledge Distillation and Masked Acoustic Modeling for Full-band Speech Restoration with Improved Intelligibilitycited by 5
- 2024DMOSpeech: Direct Metric Optimization via Distilled Diffusion Model in Zero-Shot Speech Synthesiscited by 5
- 2025LauraTSE: Target Speaker Extraction using Auto-Regressive Decoder-Only Language Modelscited by 5
- 2023Emphasized Non-Target Speaker Knowledge in Knowledge Distillation for Automatic Speaker Verificationcited by 4
- 2024Mamba-based Segmentation Model for Speaker Diarizationcited by 4
- 2025The First VoicePrivacy Attacker Challengecited by 4
- 2025MeanFlow-TSE: One-Step Generative Target Speaker Extraction with Mean Flowcited by 4
- 2025SDBench: A Comprehensive Benchmark Suite for Speaker Diarizationcited by 3
- 2025Enhancing Speaker Verification with w2v-BERT 2.0 and Knowledge Distillation guided Structured Pruningcited by 3
- 2025Adaptive Deterministic Flow Matching for Target Speaker Extractioncited by 3
- 2024Robust Channel Learning for Large-Scale Radio Speaker Verificationcited by 2
- 2024Investigation of Speaker Representation for Target-Speaker Speech Processingcited by 2
- 2025Continuous Autoregressive Modeling with Stochastic Monotonic Alignment for Speech Synthesiscited by 2
- 2025LipDiffuser: Lip-to-Speech Generation with Conditional Diffusion Modelscited by 2
- 2025NaturalVoices: A Large-Scale, Spontaneous and Emotional Podcast Dataset for Voice Conversioncited by 2
- 2026Scores Know Bobs Voice: Speaker Impersonation Attackcited by 2
- 2023Adversarial Speaker Disentanglement Using Unannotated External Data for Self-supervised Representation Based Voice Conversioncited by 1
- 2024Systematic Evaluation of Online Speaker Diarization Systems Regarding their Latencycited by 1
- 2024Flow-TSVAD: Target-Speaker Voice Activity Detection via Latent Flow Matchingcited by 1
- 2024Single-stage TTS with Masked Audio Token Modeling and Semantic Knowledge Distillationcited by 1
- 2024The First VoicePrivacy Attacker Challenge Evaluation Plancited by 1
- 2025Comparative Evaluation of Expressive Japanese Character Text-to-Speech with VITS and Style-BERT-VITS2cited by 1
- 2025Interpolating Speaker Identities in Embedding Space for Data Expansioncited by 1
- 2025Rethinking the Evaluation of Alignment Methods: Insights into Diversity, Generalisation, and Safetycited by 1
50 most cited of 97 All 97 (JSON)
- 2026Anuvani-Diarization: Zero Shot Speaker Diarization and Annotation Conventions Across 22 Languages14 Oct 2026
- 2026FASTDIAR: Frame-level speaker encoder for Streaming Diarization2 Oct 2026
- 2026Who Said What, and Will It Be Remembered? Evaluating Persistent Speaker Attribution Across Meetings30 Sep 2026
- 2026SCIC: Scope- and Codebook-Aware Instruction Conditioning for Speaker-Adapted Expressive TTS30 Sep 2026
- 2026InterBias-SV: Compound Conditions in Speaker Verification29 Sep 2026
- 2026SpeechCritic: Learning a Diagnostic Speech Judge from Limited Human Preferences28 Sep 2026
- 2026Inquesto Score: A reliability Protocol For Voice Agents24 Sep 2026
- 2026MECT: Mixture of Experts with CNN-Transformer Network for Speaker verification21 Sep 2026
- 2026COT-TTS: Audio Context-Aware Text-to-Speech with Chain-of-Thought Reasoning19 Sep 2026
- 2026X2Streaming-TTS: Causal Token-Level Text-to-Speech from Streaming Text with Speech-State Inheritance19 Aug 2026
- 2026VoiceDesigner: Text-to-Voice Generation and Editing via Unified Diffusion Modeling and Data Augmentation12 Aug 2026
- 2026DAVE: A Decoupled Audio-Visual Enhancement Framework for Real-World Speech Separation10 Aug 2026
38 more
- 2026CuteTTS: Efficient and High-Quality Speech Synthesis via Autoregressive Modeling of Continuous Latents9 Aug 2026
- 2026TIDES: A Longitudinal Bilingual Dataset for Modeling Multi-Party Social Dynamics3 Aug 2026
- 2026Breaking the Quality--Intelligibility Trade-off in Streaming Target Speaker Extraction via Deep-Feature-Anchored Preference Optimization11 Jul 2026
- 2026Diarization-Guided Qwen-ASR Adaptation for Multilingual Two-Speaker Conversational Speech9 Jul 2026
- 2026PS4: Proxy-Supervised Joint Training for Real Target Speaker Extraction9 Jul 2026
- 2026VieSpeaker: A Large-Scale Vietnamese Speaker Recognition Dataset Beyond Visual Dependency23 Jun 2026
- 2026EmotionAI: A Privacy-Preserving Computational Intelligence Pipeline for Speech-Emotion-Grounded Conversational Analysis22 Jun 2026
- 2026Kiwano: A Cutting-Edge Open-Source Toolkit for Speaker Verification21 Jun 2026
- 2026An Evaluation Framework for Text-to-Speech Voice Reconstruction19 Jun 2026
- 2026TLDR: Compressing Audio Tokens for Efficient Autoregressive Text-to-Speech8 Jun 2026
- 2026CuriosAI Submission to the CASTLE Challenge at EgoVis 202627 May 2026
- 2026Adaptive Regularization for Sparsity Control in Bregman-Based Optimizers8 May 2026
- 2026StarTSE: Towards Streaming Target Speaker Extraction via Chunk-wise Interleaved Splicing of Autoregressive Language Model21 Apr 2026
- 2026Listening Alone, Understanding Together: Collaborative Context Recovery for Privacy-Aware AI14 Apr 2026
- 2026A Benchmark and Multi-Agent System for Instruction-driven Cinematic Video Compilation12 Apr 2026
- 2026Mask2Flow-TSE: Two-Stage Target Speaker Extraction with Masking and Flow Matching13 Mar 2026
- 2026AlphaFlowTSE: One-Step Generative Target Speaker Extraction via Conditional AlphaFlow11 Mar 2026
- 2026Scores Know Bobs Voice: Speaker Impersonation Attack3 Mar 2026
- 2026Robust Long-Form Bangla Speech Processing: Automatic Speech Recognition and Speaker Diarization25 Feb 2026
- 2026DeepASMR: LLM-Based Zero-Shot ASMR Speech Generation for Anyone of Any Voice22 Jan 2026
- 2026Triage knowledge distillation for speaker verification21 Jan 2026
- 2026DAME: Duration-Aware Matryoshka Embedding for Duration-Robust Speaker Verification20 Jan 2026
- 2025MeanFlow-TSE: One-Step Generative Target Speaker Extraction with Mean Flow21 Dec 2025
- 2025Singing Timbre Popularity Assessment Based on Multimodal Large Foundation Model7 Dec 2025
- 2025NaturalVoices: A Large-Scale, Spontaneous and Emotional Podcast Dataset for Voice Conversion31 Oct 2025
- 2025Noise-Conditioned Mixture-of-Experts Framework for Robust Speaker Verification21 Oct 2025
- 2025Adaptive Deterministic Flow Matching for Target Speaker Extraction19 Oct 2025
- 2025Enhancing Speaker Verification with w2v-BERT 2.0 and Knowledge Distillation guided Structured Pruning5 Oct 2025
- 2025UniFlow-Audio: Unified Flow Matching for Audio Generation from Omni-Modalities29 Sep 2025
- 2025WEST: LLM based Speech Toolkit for Speech Understanding, Generation, and Interaction24 Sep 2025
- 2025MeanFlowSE: one-step generative speech enhancement via conditional mean flow18 Sep 2025
- 2025Towards Human-like Multimodal Conversational Agent by Generating Engaging Speech18 Sep 2025
- 2025Rethinking the Evaluation of Alignment Methods: Insights into Diversity, Generalisation, and Safety16 Sep 2025
- 2025Comparison of End-to-end Speech Assessment Models for the NOCASA 2025 Challenge3 Sep 2025
- 2025Interpolating Speaker Identities in Embedding Space for Data Expansion26 Aug 2025
- 2025SDBench: A Comprehensive Benchmark Suite for Speaker Diarization22 Jul 2025
- 2025DMOSpeech 2: Reinforcement Learning for Duration Prediction in Metric-Optimized Speech Synthesis20 Jul 2025
- 2025On Barriers to Archival Audio Processing11 Jul 2025
50 newest of 97 All 97 (JSON)
- 2024Autoregressive Diffusion Transformer for Text-to-Speech Synthesiscited by 34
- 2025F5R-TTS: Improving Flow-Matching based Text-to-Speech with Group Relative Policy Optimizationcited by 24
- 2025FlowSE: Efficient and High-Quality Speech Enhancement via Flow Matchingcited by 17
- 2023DDTSE: Discriminative Diffusion Model for Target Speech Extractioncited by 8
- 2025DMOSpeech 2: Reinforcement Learning for Duration Prediction in Metric-Optimized Speech Synthesiscited by 8
- 2024StyleTTS-ZS: Efficient High-Quality Zero-Shot Text-to-Speech Synthesis with Distilled Time-Varying Style Diffusioncited by 7
- 2025UniFlow-Audio: Unified Flow Matching for Audio Generation from Omni-Modalitiescited by 6
- 2024DMOSpeech: Direct Metric Optimization via Distilled Diffusion Model in Zero-Shot Speech Synthesiscited by 5
- 2025MeanFlow-TSE: One-Step Generative Target Speaker Extraction with Mean Flowcited by 4
- 2025Adaptive Deterministic Flow Matching for Target Speaker Extractioncited by 3
- 2025LipDiffuser: Lip-to-Speech Generation with Conditional Diffusion Modelscited by 2
- 2024Flow-TSVAD: Target-Speaker Voice Activity Detection via Latent Flow Matchingcited by 1
12 most cited of 16 All 16 (JSON)
- 2026VoiceDesigner: Text-to-Voice Generation and Editing via Unified Diffusion Modeling and Data Augmentation12 Aug 2026
- 2026CuteTTS: Efficient and High-Quality Speech Synthesis via Autoregressive Modeling of Continuous Latents9 Aug 2026
- 2026Mask2Flow-TSE: Two-Stage Target Speaker Extraction with Masking and Flow Matching13 Mar 2026
- 2026AlphaFlowTSE: One-Step Generative Target Speaker Extraction via Conditional AlphaFlow11 Mar 2026
- 2025MeanFlow-TSE: One-Step Generative Target Speaker Extraction with Mean Flow21 Dec 2025
- 2025Adaptive Deterministic Flow Matching for Target Speaker Extraction19 Oct 2025
- 2025UniFlow-Audio: Unified Flow Matching for Audio Generation from Omni-Modalities29 Sep 2025
- 2025DMOSpeech 2: Reinforcement Learning for Duration Prediction in Metric-Optimized Speech Synthesis20 Jul 2025
- 2025FlowSE: Efficient and High-Quality Speech Enhancement via Flow Matching26 May 2025
- 2025LipDiffuser: Lip-to-Speech Generation with Conditional Diffusion Models16 May 2025
- 2025F5R-TTS: Improving Flow-Matching based Text-to-Speech with Group Relative Policy Optimization3 Apr 2025
- 2024DMOSpeech: Direct Metric Optimization via Distilled Diffusion Model in Zero-Shot Speech Synthesis14 Oct 2024
12 newest of 16 All 16 (JSON)
- 2024Joint Semantic Knowledge Distillation and Masked Acoustic Modeling for Full-band Speech Restoration with Improved Intelligibilitycited by 5
- 2023Emphasized Non-Target Speaker Knowledge in Knowledge Distillation for Automatic Speaker Verificationcited by 4
- 2025Enhancing Speaker Verification with w2v-BERT 2.0 and Knowledge Distillation guided Structured Pruningcited by 3
- 2024Single-stage TTS with Masked Audio Token Modeling and Semantic Knowledge Distillationcited by 1
- 2026Triage knowledge distillation for speaker verificationcited by 0
- 2026Triage knowledge distillation for speaker verification21 Jan 2026
- 2025Enhancing Speaker Verification with w2v-BERT 2.0 and Knowledge Distillation guided Structured Pruning5 Oct 2025
- 2024Single-stage TTS with Masked Audio Token Modeling and Semantic Knowledge Distillation17 Sep 2024
- 2024Joint Semantic Knowledge Distillation and Masked Acoustic Modeling for Full-band Speech Restoration with Improved Intelligibility14 Sep 2024
- 2023Emphasized Non-Target Speaker Knowledge in Knowledge Distillation for Automatic Speaker Verification26 Sep 2023
- 2025Rethinking the Evaluation of Alignment Methods: Insights into Diversity, Generalisation, and Safetycited by 1
- 2025Seewo's Submission to MLC-SLM: Lessons learned from Speech Reasoning Language Modelscited by 0
- 2026COT-TTS: Audio Context-Aware Text-to-Speech with Chain-of-Thought Reasoningcited by 0
- 2026COT-TTS: Audio Context-Aware Text-to-Speech with Chain-of-Thought Reasoning19 Sep 2026
- 2025Rethinking the Evaluation of Alignment Methods: Insights into Diversity, Generalisation, and Safety16 Sep 2025
- 2025Seewo's Submission to MLC-SLM: Lessons learned from Speech Reasoning Language Models16 Jun 2025
- 2024Mamba-based Segmentation Model for Speaker Diarizationcited by 4
- 2024Mamba-based Segmentation Model for Speaker Diarization9 Oct 2024
- 2025Spark-TTS: An Efficient LLM-Based Text-to-Speech Model with Single-Stream Decoupled Speech Tokenscited by 113
- 2023RepCodec: A Speech Representation Codec for Speech Tokenizationcited by 27
- 2024MaskSR: Masked Language Model for Full-band Speech Restorationcited by 24
- 2024ESPnet-SPK: full pipeline speaker embedding toolkit with reproducible recipes, self-supervised front-ends, and off-the-shelf modelscited by 22
- 2025LLaSE-G1: Incentivizing Generalization Capability for LLaMA-based Speech Enhancementcited by 19
- 2024Coding Speech through Vocal Tract Kinematicscited by 18
- 2024TTSDS -- Text-to-Speech Distribution Scorecited by 14
- 2023Disentangling Voice and Content with Self-Supervision for Speaker Recognitioncited by 13
- 2024What Do Self-Supervised Speech and Speaker Models Learn? New Findings From a Cross Model Layer-Wise Analysiscited by 11
- 2024TSELM: Target Speaker Extraction using Discrete Tokens and Language Modelscited by 10
- 2024TouchTTS: An Embarrassingly Simple TTS Framework that Everyone Can Touchcited by 10
- 2025Voila: Voice-Language Foundation Models for Real-Time Autonomous Interaction and Voice Role-Playcited by 10
12 most cited of 67 All 67 (JSON)
- 2026Anuvani-Diarization: Zero Shot Speaker Diarization and Annotation Conventions Across 22 Languages14 Oct 2026
- 2026FASTDIAR: Frame-level speaker encoder for Streaming Diarization2 Oct 2026
- 2026Who Said What, and Will It Be Remembered? Evaluating Persistent Speaker Attribution Across Meetings30 Sep 2026
- 2026SCIC: Scope- and Codebook-Aware Instruction Conditioning for Speaker-Adapted Expressive TTS30 Sep 2026
- 2026InterBias-SV: Compound Conditions in Speaker Verification29 Sep 2026
- 2026Inquesto Score: A reliability Protocol For Voice Agents24 Sep 2026
- 2026X2Streaming-TTS: Causal Token-Level Text-to-Speech from Streaming Text with Speech-State Inheritance19 Aug 2026
- 2026DAVE: A Decoupled Audio-Visual Enhancement Framework for Real-World Speech Separation10 Aug 2026
- 2026TIDES: A Longitudinal Bilingual Dataset for Modeling Multi-Party Social Dynamics3 Aug 2026
- 2026Breaking the Quality--Intelligibility Trade-off in Streaming Target Speaker Extraction via Deep-Feature-Anchored Preference Optimization11 Jul 2026
- 2026Diarization-Guided Qwen-ASR Adaptation for Multilingual Two-Speaker Conversational Speech9 Jul 2026
- 2026PS4: Proxy-Supervised Joint Training for Real Target Speaker Extraction9 Jul 2026
12 newest of 67 All 67 (JSON)