# Vocal Style Factorization for Effective Speaker Recognition in Affective Scenarios

arXiv: https://arxiv.org/abs/2305.07997 · HTML: https://mlsanity.com/paper/2305.07997/

## Key facts

- No citing papers in the map yet (data through 19 Oct 2026).
- Placed in Emotional speech synthesis and recognition, one of 11 directions in Speech and audio language models.
- Builds on 14 of its 35 references that are in the map.
- First posted on arXiv 13 May 2023.

## Placed in

- Speech and audio language models / [Emotional speech synthesis and recognition](https://mlsanity.com/speech/emotional-speech-synthesis-and-recognition/)

## Builds on (most cited first, 14 of 14)

- [ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification](https://mlsanity.com/paper/2005.07143/) (2020)
- [Self-Normalizing Neural Networks](https://mlsanity.com/paper/1706.02515/) (2017)
- [SpeechBrain: A General-Purpose Speech Toolkit](https://mlsanity.com/paper/2106.04624/) (2021)
- [Generalized End-to-End Loss for Speaker Verification](https://mlsanity.com/paper/1710.10467/) (2017)
- [Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesis](https://mlsanity.com/paper/1803.09017/) (2018)
- [Speaker Recognition from Raw Waveform with SincNet](https://mlsanity.com/paper/1808.00158/) (2018)
- [MFA-Conformer: Multi-scale Feature Aggregation Conformer for Automatic Speaker Verification](https://mlsanity.com/paper/2203.15249/) (2022)
- [TitaNet: Neural Model for speaker representation with 1D Depth-wise separable convolutions and global context](https://mlsanity.com/paper/2110.04410/) (2021)
- [x-vectors meet emotions: A study on dependencies between emotion and speaker recognition](https://mlsanity.com/paper/2002.05039/) (2020)
- [Speech Synthesis with Mixed Emotions](https://mlsanity.com/paper/2208.05890/) (2022)
- [The 2021 NIST Speaker Recognition Evaluation](https://mlsanity.com/paper/2204.10242/) (2022)
- [Speaker-invariant Affective Representation Learning via Adversarial Training](https://mlsanity.com/paper/1911.01533/) (2019)
- [DeepTalk: Vocal Style Encoding for Speaker Recognition and Speech Synthesis](https://mlsanity.com/paper/2012.05084/) (2020)
- [Emotional Speaker Identification using a Novel Capsule Nets Model](https://mlsanity.com/paper/2201.02994/) (2022)

## Abstract (arXiv)

The accuracy of automated speaker recognition is negatively impacted by change in emotions in a person's speech. In this paper, we hypothesize that speaker identity is composed of various vocal style factors that may be learned from unlabeled data and re-combined using a neural network to generate a holistic speaker identity representation for affective scenarios. In this regard, we propose the E-Vector architecture, composed of a 1-D CNN for learning speaker identity features and a vocal style factorization technique for determining vocal styles. Experiments conducted on the MSP-Podcast dataset demonstrate that the proposed architecture improves state-of-the-art speaker recognition accuracy in the affective domain over baseline ECAPA-TDNN speaker recognition models. For instance, the true match rate at a false match rate of 1% improves from 27.6% to 46.2%.