# Xi+: Uncertainty Supervision for Robust Speaker Embedding

arXiv: https://arxiv.org/abs/2509.05993 · HTML: https://mlsanity.com/paper/2509.05993/

## Key facts

- Cited by 5 papers in the map (data through 19 Oct 2026).
- Placed in Speaker recognition and diarization, one of 11 directions in Speech and audio language models.
- Builds on 19 of its 29 references that are in the map.
- First posted on arXiv 7 Sep 2025.

## Placed in

- Speech and audio language models / [Speaker recognition and diarization](https://mlsanity.com/speech/speaker-recognition-and-diarization/)

## Builds on (most cited first, 19 of 19)

- [VoxCeleb2: Deep Speaker Recognition](https://mlsanity.com/paper/1806.05622/) (2018)
- [VoxCeleb: a large-scale speaker identification dataset](https://mlsanity.com/paper/1706.08612/) (2017)
- [ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification](https://mlsanity.com/paper/2005.07143/) (2020)
- [MUSAN: A Music, Speech, and Noise Corpus](https://mlsanity.com/paper/1510.08484/) (2015)
- [Wespeaker: A Research and Production oriented Speaker Embedding Learning Toolkit](https://mlsanity.com/paper/2210.17016/) (2022)
- [Probabilistic Face Embeddings](https://mlsanity.com/paper/1904.09658/) (2019)
- [Data Uncertainty Learning in Face Recognition](https://mlsanity.com/paper/2003.11339/) (2020)
- [Speaker Recognition Based on Deep Learning: An Overview](https://mlsanity.com/paper/2012.00931/) (2020)
- [The IDLAB VoxSRC-20 Submission: Large Margin Fine-Tuning and Quality-Aware Score Calibration in DNN Based Speaker Verification](https://mlsanity.com/paper/2010.11255/) (2020)
- [Reshape Dimensions Network for Speaker Recognition](https://mlsanity.com/paper/2407.18223/) (2024)
- [MAP: Multimodal Uncertainty-Aware Vision-Language Pre-training Model](https://mlsanity.com/paper/2210.05335/) (2022)
- [rVAD: An Unsupervised Segment-Based Robust Voice Activity Detection Method](https://mlsanity.com/paper/1906.03588/) (2019)
- [Disentangling Voice and Content with Self-Supervision for Speaker Recognition](https://mlsanity.com/paper/2310.01128/) (2023)
- [Xi-Vector Embedding for Speaker Recognition](https://mlsanity.com/paper/2108.05679/) (2021)
- [Golden Gemini is All You Need: Finding the Sweet Spots for Speaker Verification](https://mlsanity.com/paper/2312.03620/) (2023)
- [Overview of Speaker Modeling and Its Applications: From the Lens of Deep Speaker Representation Learning](https://mlsanity.com/paper/2407.15188/) (2024)
- [Incorporating Uncertainty from Speaker Embedding Estimation to Speaker Verification](https://mlsanity.com/paper/2302.11763/) (2023)
- [Cosine Scoring with Uncertainty for Neural Speaker Embedding](https://mlsanity.com/paper/2403.06404/) (2024)
- [Analysis of ABC Frontend Audio Systems for the NIST-SRE24](https://mlsanity.com/paper/2505.15320/) (2025)

## Built on by (most cited first, 5 of 5)

- [U3-xi: Pushing the Boundaries of Speaker Recognition by Incorporating Uncertainty](https://mlsanity.com/paper/2601.15719/) (2026)
- [Towards Robust Uncertainty-Aware Speaker Modeling](https://mlsanity.com/paper/2607.04937/) (2026)
- [A Unified Uncertainty-Aware Back-End for Speaker Verification: Scoring, Normalization, and Calibration](https://mlsanity.com/paper/2609.01221/) (2026)
- [Entropy-aware logistic regression for fusion of large-scale speaker recognition systems](https://mlsanity.com/paper/2609.23727/) (2026)
- [CAFE: Confining Feature Distillation to Language-Routed Adapters for Compact Multilingual Automatic Speech Recognition](https://mlsanity.com/paper/2610.19134/) (2026)

## Abstract (arXiv)

There are various factors that can influence the performance of speaker recognition systems, such as emotion, language and other speaker-related or context-related variations. Since individual speech frames do not contribute equally to the utterance-level representation, it is essential to estimate the importance or reliability of each frame. The xi-vector model addresses this by assigning different weights to frames based on uncertainty estimation. However, its uncertainty estimation model is implicitly trained through classification loss alone and does not consider the temporal relationships between frames, which may lead to suboptimal supervision. In this paper, we propose an improved architecture, xi+. Compared to xi-vector, xi+ incorporates a temporal attention module to capture frame-level uncertainty in a context-aware manner. In addition, we introduce a novel loss function, Stochastic Variance Loss, which explicitly supervises the learning of uncertainty. Results demonstrate consistent performance improvements of about 10% on the VoxCeleb1-O set and 11% on the NIST SRE 2024 evaluation set.