mlsanityData to 19 Oct 2026

Vocal Style Factorization for Effective Speaker Recognition in Affective Scenarios

arXiv 2305.07997 ↗First posted 13 May 2023eess.AS cs.SDcited by 0

On the map: Speech and audio language models → Emotional speech synthesis and recognition

Abstract (arXiv)

The accuracy of automated speaker recognition is negatively impacted by change in emotions in a person's speech. In this paper, we hypothesize that speaker identity is composed of various vocal style factors that may be learned from unlabeled data and re-combined using a neural network to generate a holistic speaker identity representation for affective scenarios. In this regard, we propose the E-Vector architecture, composed of a 1-D CNN for learning speaker identity features and a vocal style factorization technique for determining vocal styles. Experiments conducted on the MSP-Podcast dataset demonstrate that the proposed architecture improves state-of-the-art speaker recognition accuracy in the affective domain over baseline ECAPA-TDNN speaker recognition models. For instance, the true match rate at a false match rate of 1% improves from 27.6% to 46.2%.

Key facts. No citing papers in the map yet (data through 19 Oct 2026). Placed in Emotional speech synthesis and recognition, one of 11 directions in Speech and audio language models. Builds on 14 of its 35 references that are in the map. First posted on arXiv 13 May 2023.

Builds on

14 of 35 references.

Closest ancestors · 1

Same direction, newest first

  1. 2022Speech Synthesis with Mixed Emotions11 Aug

Other references · 13

  1. 2020ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verificationcited by 980
  2. 2017Self-Normalizing Neural Networkscited by 603
  3. 2021SpeechBrain: A General-Purpose Speech Toolkitcited by 532
  4. 2017Generalized End-to-End Loss for Speaker Verificationcited by 376
  5. 2018Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesiscited by 364
  6. 2018Speaker Recognition from Raw Waveform with SincNetcited by 205
  7. 2022MFA-Conformer: Multi-scale Feature Aggregation Conformer for Automatic Speaker Verificationcited by 91
  8. 2021TitaNet: Neural Model for speaker representation with 1D Depth-wise separable convolutions and global contextcited by 87
  9. 2020x-vectors meet emotions: A study on dependencies between emotion and speaker recognitioncited by 45
  10. 2022The 2021 NIST Speaker Recognition Evaluationcited by 21
  11. 2019Speaker-invariant Affective Representation Learning via Adversarial Trainingcited by 17
  12. 2020DeepTalk: Vocal Style Encoding for Speaker Recognition and Speech Synthesiscited by 3
1 more
  1. 2022Emotional Speaker Identification using a Novel Capsule Nets Modelcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2022Is Style All You Need? Dependencies Between Emotion and GST-based Speaker Recognition
  2. 2023EmoDiarize: Speaker Diarization and Emotion Identification from Speech Signals using Convolutional Neural Networks
  3. 2024Improving speaker verification robustness with synthetic emotional utterances
  4. 2025The Affective Bridge: Preserving Speech Representations while Enhancing Deepfake Detection vian emotional Constraints
  5. 2026EmotionAI: A Privacy-Preserving Computational Intelligence Pipeline for Speech-Emotion-Grounded Conversational Analysis
  6. 2023Transforming the Embeddings: A Lightweight Technique for Speech Emotion Recognition Tasks
  7. 2025Learning Emotion-Invariant Speaker Representations for Speaker Verification
  8. 2026VISAFF: Speaker-Centered Visual Affective Feature Learning for Emotion Recognition in Conversation