# Margin Matters: Towards More Discriminative Deep Neural Network Embeddings for Speaker Recognition

arXiv: https://arxiv.org/abs/1906.07317 · HTML: https://mlsanity.com/paper/1906.07317/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jun 2019.

## Built on by (most cited first, 27 of 27)

- [Adversarial defense for automatic speaker verification by cascaded self-supervised learning models](https://mlsanity.com/paper/2102.07047/) (2021)
- [Improving the Adversarial Robustness for Speaker Verification by Self-Supervised Learning](https://mlsanity.com/paper/2106.00273/) (2021)
- [Dr-Vectors: Decision Residual Networks and an Improved Loss for Speaker Recognition](https://mlsanity.com/paper/2104.01989/) (2021)
- [Multi-task Voice Activated Framework using Self-supervised Learning](https://mlsanity.com/paper/2110.01077/) (2021)
- [Leveraging ASR Pretrained Conformers for Speaker Verification through Transfer Learning and Knowledge Distillation](https://mlsanity.com/paper/2309.03019/) (2023)
- [EfficientTDNN: Efficient Architecture Search for Speaker Recognition](https://mlsanity.com/paper/2103.13581/) (2021)
- [Real Additive Margin Softmax for Speaker Verification](https://mlsanity.com/paper/2110.09116/) (2021)
- [Audio Anti-spoofing Using a Simple Attention Module and Joint Optimization Based on Additive Angular Margin Loss and Meta-learning](https://mlsanity.com/paper/2211.09898/) (2022)
- [MACCIF-TDNN: Multi aspect aggregation of channel and context interdependence features in TDNN-based speaker verification](https://mlsanity.com/paper/2107.03104/) (2021)
- [Learning Audio-Visual embedding for Person Verification in the Wild](https://mlsanity.com/paper/2209.04093/) (2022)
- [You Are What You Say: Exploiting Linguistic Content for VoicePrivacy Attacks](https://mlsanity.com/paper/2506.09521/) (2025)
- [DS-TDNN: Dual-stream Time-delay Neural Network with Global-aware Filter for Speaker Verification](https://mlsanity.com/paper/2303.11020/) (2023)
- [Phonetic-aware speaker embedding for far-field speaker verification](https://mlsanity.com/paper/2311.15627/) (2023)
- [Adversarial Data Augmentation for Robust Speaker Verification](https://mlsanity.com/paper/2402.02699/) (2024)
- [Speech After Gender: A Trans-Feminine Perspective on Next Steps for Speech Science and Technology](https://mlsanity.com/paper/2407.07235/) (2024)
- [Speaker Retrieval in the Wild: Challenges, Effectiveness and Robustness](https://mlsanity.com/paper/2504.18950/) (2025)
- [Leveraging Speaker Embeddings with Adversarial Multi-task Learning for Age Group Classification](https://mlsanity.com/paper/2301.09058/) (2023)
- [Cluster-Guided Unsupervised Domain Adaptation for Deep Speaker Embedding](https://mlsanity.com/paper/2303.15944/) (2023)
- [DASA: Difficulty-Aware Semantic Augmentation for Speaker Verification](https://mlsanity.com/paper/2310.12111/) (2023)
- [Query-by-Example Keyword Spotting Using Spectral-Temporal Graph Attentive Pooling and Multi-Task Learning](https://mlsanity.com/paper/2409.00099/) (2024)
- [Memory-Efficient Training for Text-Dependent SV with Independent Pre-trained Models](https://mlsanity.com/paper/2411.10828/) (2024)
- [Memory-Efficient Training for Deep Speaker Embedding Learning in Speaker Verification](https://mlsanity.com/paper/2412.01195/) (2024)
- [Task-Specific Audio Coding for Machines: Machine-Learned Latent Features Are Codes for That Machine](https://mlsanity.com/paper/2507.12701/) (2025)
- [Alpha Divergence Losses for Biometric Verification](https://mlsanity.com/paper/2511.13621/) (2025)
- [Adaptive Regularization for Sparsity Control in Bregman-Based Optimizers](https://mlsanity.com/paper/2605.07892/) (2026)
- [Exploiting Neural Audio Codec Latents for Adversarial Audio Attacks](https://mlsanity.com/paper/2606.20893/) (2026)
- [Sparsity-Inducing Divergence Losses for Biometric Verification](https://mlsanity.com/paper/2606.31664/) (2026)

## Abstract (arXiv)

Recently, speaker embeddings extracted from a speaker discriminative deep neural network (DNN) yield better performance than the conventional methods such as i-vector. In most cases, the DNN speaker classifier is trained using cross entropy loss with softmax. However, this kind of loss function does not explicitly encourage inter-class separability and intra-class compactness. As a result, the embeddings are not optimal for speaker recognition tasks. In this paper, to address this issue, three different margin based losses which not only separate classes but also demand a fixed margin between classes are introduced to deep speaker embedding learning. It could be demonstrated that the margin is the key to obtain more discriminative speaker embeddings. Experiments are conducted on two public text independent tasks: VoxCeleb1 and Speaker in The Wild (SITW). The proposed approach can achieve the state-of-the-art performance, with 25% ~ 30% equal error rate (EER) reduction on both tasks when compared to strong baselines using cross entropy loss with softmax, obtaining 2.238% EER on VoxCeleb1 test set and 2.761% EER on SITW core-core test set, respectively.