# Deep Speaker: an End-to-End Neural Speaker Embedding System

arXiv: https://arxiv.org/abs/1705.02304 · HTML: https://mlsanity.com/paper/1705.02304/

## Key facts

- Cited by 58 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 May 2017.

## Built on by (most cited first, 30 of 58)

- [AdaSpeech: Adaptive Text to Speech for Custom Voice](https://mlsanity.com/paper/2103.00993/) (2021)
- [Turn-to-Diarize: Online Speaker Diarization Constrained by Transformer Transducer Speaker Turn Detection](https://mlsanity.com/paper/2109.11641/) (2021)
- [Learning Efficient Representations for Keyword Spotting with Triplet Loss](https://mlsanity.com/paper/2101.04792/) (2021)
- [Towards Understanding and Mitigating Audio Adversarial Examples for Speaker Recognition](https://mlsanity.com/paper/2206.03393/) (2022)
- [Bias in Automated Speaker Recognition](https://mlsanity.com/paper/2201.09486/) (2022)
- [STYLER: Style Factor Modeling with Rapidity and Robustness via Speech Decomposition for Expressive and Controllable Neural Text to Speech](https://mlsanity.com/paper/2103.09474/) (2021)
- [Detecting Individual Decision-Making Style: Exploring Behavioral Stylometry in Chess](https://mlsanity.com/paper/2208.01366/) (2022)
- [AdaSpeech 2: Adaptive Text to Speech with Untranscribed Data](https://mlsanity.com/paper/2104.09715/) (2021)
- [DiarizationLM: Speaker Diarization Post-Processing with Large Language Models](https://mlsanity.com/paper/2401.03506/) (2024)
- [Lexical Speaker Error Correction: Leveraging Language Models for Speaker Diarization Error Correction](https://mlsanity.com/paper/2306.09313/) (2023)
- [CM-TTS: Enhancing Real Time Text-to-Speech Synthesis Efficiency through Weighted Samplers and Consistency Models](https://mlsanity.com/paper/2404.00569/) (2024)
- [MFA: TDNN with Multi-scale Frequency-channel Attention for Text-independent Speaker Verification with Short Utterances](https://mlsanity.com/paper/2202.01624/) (2022)
- [Dr-Vectors: Decision Residual Networks and an Improved Loss for Speaker Recognition](https://mlsanity.com/paper/2104.01989/) (2021)
- [AS2T: Arbitrary Source-To-Target Adversarial Attack on Speaker Recognition Systems](https://mlsanity.com/paper/2206.03351/) (2022)
- [A Method to Reveal Speaker Identity in Distributed ASR Training, and How to Counter It](https://mlsanity.com/paper/2104.07815/) (2021)
- [Camera-Conditioned Stable Feature Generation for Isolated Camera Supervised Person Re-IDentification](https://mlsanity.com/paper/2203.15210/) (2022)
- [ADAPTERMIX: Exploring the Efficacy of Mixture of Adapters for Low-Resource TTS Adaptation](https://mlsanity.com/paper/2305.18028/) (2023)
- [ECAPA-TDNN for Multi-speaker Text-to-speech Synthesis](https://mlsanity.com/paper/2203.10473/) (2022)
- [Evaluating Parameter-Efficient Transfer Learning Approaches on SURE Benchmark for Speech Understanding](https://mlsanity.com/paper/2303.03267/) (2023)
- [HEFT: Homomorphically Encrypted Fusion of Biometric Templates](https://mlsanity.com/paper/2208.07241/) (2022)
- [Catch You and I Can: Revealing Source Voiceprint Against Voice Conversion](https://mlsanity.com/paper/2302.12434/) (2023)
- [Label Aware Speech Representation Learning For Language Identification](https://mlsanity.com/paper/2306.04374/) (2023)
- [Voice Cloning: Comprehensive Survey](https://mlsanity.com/paper/2505.00579/) (2025)
- [Fusion of Embeddings Networks for Robust Combination of Text Dependent and Independent Speaker Recognition](https://mlsanity.com/paper/2106.10169/) (2021)
- [SEC4SR: A Security Analysis Platform for Speaker Recognition](https://mlsanity.com/paper/2109.01766/) (2021)
- [Real Additive Margin Softmax for Speaker Verification](https://mlsanity.com/paper/2110.09116/) (2021)
- [Robust Vocal Quality Feature Embeddings for Dysphonic Voice Detection](https://mlsanity.com/paper/2211.09858/) (2022)
- [CryCeleb: A Speaker Verification Dataset Based on Infant Cry Sounds](https://mlsanity.com/paper/2305.00969/) (2023)
- [Learning Repeatable Speech Embeddings Using An Intra-class Correlation Regularizer](https://mlsanity.com/paper/2310.17049/) (2023)
- [Speaker attribution with voice profiles by graph-based semi-supervised learning](https://mlsanity.com/paper/2102.03634/) (2021)

## Abstract (arXiv)

We present Deep Speaker, a neural speaker embedding system that maps utterances to a hypersphere where speaker similarity is measured by cosine similarity. The embeddings generated by Deep Speaker can be used for many tasks, including speaker identification, verification, and clustering. We experiment with ResCNN and GRU architectures to extract the acoustic features, then mean pool to produce utterance-level speaker embeddings, and train using triplet loss based on cosine similarity. Experiments on three distinct datasets suggest that Deep Speaker outperforms a DNN-based i-vector baseline. For example, Deep Speaker reduces the verification equal error rate by 50% (relatively) and improves the identification accuracy by 60% (relatively) on a text-independent dataset. We also present results that suggest adapting from a model trained with Mandarin can improve accuracy for English speaker recognition.