# Is Style All You Need? Dependencies Between Emotion and GST-based Speaker Recognition

arXiv: https://arxiv.org/abs/2211.08213 · HTML: https://mlsanity.com/paper/2211.08213/

## Key facts

- No citing papers in the map yet (data through 19 Oct 2026).
- Placed in Emotional speech synthesis and recognition, one of 11 directions in Speech and audio language models.
- Builds on 6 of its 22 references that are in the map.
- First posted on arXiv 15 Nov 2022.

## Placed in

- Speech and audio language models / [Emotional speech synthesis and recognition](https://mlsanity.com/speech/emotional-speech-synthesis-and-recognition/)

## Builds on (most cited first, 6 of 6)

- [ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification](https://mlsanity.com/paper/2005.07143/) (2020)
- [SpeechBrain: A General-Purpose Speech Toolkit](https://mlsanity.com/paper/2106.04624/) (2021)
- [Emotion Recognition in Conversation: Research Challenges, Datasets, and Recent Advances](https://mlsanity.com/paper/1905.02947/) (2019)
- [x-vectors meet emotions: A study on dependencies between emotion and speaker recognition](https://mlsanity.com/paper/2002.05039/) (2020)
- [CopyPaste: An Augmentation Method for Speech Emotion Recognition](https://mlsanity.com/paper/2010.14602/) (2020)
- [DeepTalk: Vocal Style Encoding for Speaker Recognition and Speech Synthesis](https://mlsanity.com/paper/2012.05084/) (2020)

## Abstract (arXiv)

In this work, we study the hypothesis that speaker identity embeddings extracted from speech samples may be used for detection and classification of emotion. In particular, we show that emotions can be effectively identified by learning speaker identities by use of a 1-D Triplet Convolutional Neural Network (CNN) & Global Style Token (GST) scheme (e.g., DeepTalk Network) and reusing the trained speaker recognition model weights to generate features in the emotion classification domain. The automatic speaker recognition (ASR) network is trained with VoxCeleb1, VoxCeleb2, and Librispeech datasets with a triplet training loss function using speaker identity labels. Using an Support Vector Machine (SVM) classifier, we map speaker identity embeddings into discrete emotion categories from the CREMA-D, IEMOCAP, and MSP-Podcast datasets. On the task of speech emotion detection, we obtain 80.8% ACC with acted emotion samples from CREMA-D, 81.2% ACC with semi-natural emotion samples in IEMOCAP, and 66.9% ACC with natural emotion samples in MSP-Podcast. We also propose a novel two-stage hierarchical classifier (HC) approach which demonstrates +2% ACC improvement on CREMA-D emotion samples. Through this work, we seek to convey the importance of holistically modeling intra-user variation within audio samples