# Speaker Embeddings as Individuality Proxy for Voice Stress Detection

arXiv: https://arxiv.org/abs/2306.05915 · HTML: https://mlsanity.com/paper/2306.05915/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 31 references that are in the map.
- First posted on arXiv 9 Jun 2023.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [CvT: Introducing Convolutions to Vision Transformers](https://mlsanity.com/paper/2103.15808/) (2021)
- [ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification](https://mlsanity.com/paper/2005.07143/) (2020)
- [SpeechBrain: A General-Purpose Speech Toolkit](https://mlsanity.com/paper/2106.04624/) (2021)
- [Generalized End-to-End Loss for Speaker Verification](https://mlsanity.com/paper/1710.10467/) (2017)
- [CREPE: A Convolutional Representation for Pitch Estimation](https://mlsanity.com/paper/1802.06182/) (2018)
- [BYOL for Audio: Self-Supervised Learning for General-Purpose Audio Representation](https://mlsanity.com/paper/2103.06695/) (2021)
- [HEAR: Holistic Evaluation of Audio Representations](https://mlsanity.com/paper/2203.03022/) (2022)
- [Masked Spectrogram Modeling using Masked Autoencoders for Learning General-purpose Audio Representation](https://mlsanity.com/paper/2204.12260/) (2022)
- [SERAB: A multi-lingual benchmark for speech emotion recognition](https://mlsanity.com/paper/2110.03414/) (2021)
- [Investigating Self-supervised Pretraining Frameworks for Pathological Speech Recognition](https://mlsanity.com/paper/2203.15431/) (2022)
- [On the Effect of Observed Subject Biases in Apparent Personality Analysis from Audio-visual Signals](https://mlsanity.com/paper/1909.05568/) (2019)
- [BYOL-S: Learning Self-supervised Speech Representations by Bootstrapping](https://mlsanity.com/paper/2206.12038/) (2022)
- [Experimental investigation on STFT phase representations for deep learning-based dysarthric speech detection](https://mlsanity.com/paper/2110.03283/) (2021)
- [Hybrid Handcrafted and Learnable Audio Representation for Analysis of Speech Under Cognitive and Physical Load](https://mlsanity.com/paper/2203.16637/) (2022)
- [Efficient Speech Quality Assessment using Self-supervised Framewise Embeddings](https://mlsanity.com/paper/2211.06646/) (2022)
- [Personalized Task Load Prediction in Speech Communication](https://mlsanity.com/paper/2303.00630/) (2023)

## Abstract (arXiv)

Since the mental states of the speaker modulate speech, stress introduced by cognitive or physical loads could be detected in the voice. The existing voice stress detection benchmark has shown that the audio embeddings extracted from the Hybrid BYOL-S self-supervised model perform well. However, the benchmark only evaluates performance separately on each dataset, but does not evaluate performance across the different types of stress and different languages. Moreover, previous studies found strong individual differences in stress susceptibility. This paper presents the design and development of voice stress detection, trained on more than 100 speakers from 9 language groups and five different types of stress. We address individual variabilities in voice stress analysis by adding speaker embeddings to the hybrid BYOL-S features. The proposed method significantly improves voice stress detection performance with an input audio length of only 3-5 seconds.