# Post-Training Embedding Alignment for Decoupling Enrollment and Runtime Speaker Recognition Models

arXiv: https://arxiv.org/abs/2401.12440 · HTML: https://mlsanity.com/paper/2401.12440/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 17 references that are in the map.
- First posted on arXiv 23 Jan 2024.

## Builds on (most cited first, 9 of 9)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Generalized End-to-End Loss for Speaker Verification](https://mlsanity.com/paper/1710.10467/) (2017)
- [SphereFace2: Binary Classification is All You Need for Deep Face Recognition](https://mlsanity.com/paper/2108.01513/) (2021)
- [Dr-Vectors: Decision Residual Networks and an Improved Loss for Speaker Recognition](https://mlsanity.com/paper/2104.01989/) (2021)
- [Towards Lightweight Applications: Asymmetric Enroll-Verify Structure for Speaker Verification](https://mlsanity.com/paper/2110.04438/) (2021)
- [STC speaker recognition systems for the NIST SRE 2021](https://mlsanity.com/paper/2111.02298/) (2021)
- [Analyzing speaker verification embedding extractors and back-ends under language and channel mismatch](https://mlsanity.com/paper/2203.10300/) (2022)
- [Fusion of Embeddings Networks for Robust Combination of Text Dependent and Independent Speaker Recognition](https://mlsanity.com/paper/2106.10169/) (2021)
- [Universal speaker recognition encoders for different speech segments duration](https://mlsanity.com/paper/2210.16231/) (2022)

## Abstract (arXiv)

Automated speaker identification (SID) is a crucial step for the personalization of a wide range of speech-enabled services. Typical SID systems use a symmetric enrollment-verification framework with a single model to derive embeddings both offline for voice profiles extracted from enrollment utterances, and online from runtime utterances. Due to the distinct circumstances of enrollment and runtime, such as different computation and latency constraints, several applications would benefit from an asymmetric enrollment-verification framework that uses different models for enrollment and runtime embedding generation. To support this asymmetric SID where each of the two models can be updated independently, we propose using a lightweight neural network to map the embeddings from the two independent models to a shared speaker embedding space. Our results show that this approach significantly outperforms cosine scoring in a shared speaker logit space for models that were trained with a contrastive loss on large datasets with many speaker identities. This proposed Neural Embedding Speaker Space Alignment (NESSA) combined with an asymmetric update of only one of the models delivers at least 60% of the performance gain achieved by updating both models in the standard symmetric SID approach.