mlsanityData to 7 Oct 2026

Learning Speech Emotion Representations in the Quaternion Domain

arXiv 2204.02385 ↗First posted 5 Apr 2022eess.AS cs.LG cs.SDcited by 7

Not on the map. Why?

Abstract (arXiv)

The modeling of human emotion expression in speech signals is an important, yet challenging task. The high resource demand of speech emotion recognition models, combined with the the general scarcity of emotion-labelled data are obstacles to the development and application of effective solutions in this field. In this paper, we present an approach to jointly circumvent these difficulties. Our method, named RH-emo, is a novel semi-supervised architecture aimed at extracting quaternion embeddings from real-valued monoaural spectrograms, enabling the use of quaternion-valued networks for speech emotion recognition tasks. RH-emo is a hybrid real/quaternion autoencoder network that consists of a real-valued encoder in parallel to a real-valued emotion classifier and a quaternion-valued decoder. On the one hand, the classifier permits to optimize each latent axis of the embeddings for the classification of a specific emotion-related characteristic: valence, arousal, dominance and overall emotion. On the other hand, the quaternion reconstruction enables the latent dimension to develop intra-channel correlations that are required for an effective representation as a quaternion entity. We test our approach on speech emotion recognition tasks using four popular datasets: Iemocap, Ravdess, EmoDb and Tess, comparing the performance of three well-established real-valued CNN architectures (AlexNet, ResNet-50, VGG) and their quaternion-valued equivalent fed with the embeddings created with RH-emo. We obtain a consistent improvement in the test accuracy for all datasets, while drastically reducing the resources' demand of models. Moreover, we performed additional experiments and ablation studies that confirm the effectiveness of our approach. The RH-emo repository is available at: https://github.com/ispamm/rhemo.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 25 of its 75 references that are in the map. First posted on arXiv 5 Apr 2022.

Builds on

25 of 75 references.

References · 25

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2021Emotion Recognition from Speech Using Wav2vec 2.0 Embeddingscited by 158
  5. 2021EmoBERTa: Speaker-Aware Emotion Recognition in Conversation with RoBERTacited by 59
  6. 2017Deep Quaternion Networkscited by 48
  7. 2019Generating Synthetic Audio Data for Attention-Based Speech Recognition Systemscited by 46
  8. 2020You Do Not Need More Data: Improving End-To-End Speech Recognition by Text-To-Speech Data Augmentationcited by 29
  9. 2019Lightweight and Efficient Neural Natural Language Processing with Quaternion Networkscited by 26
  10. 2021PHNNs: Lightweight Neural Networks via Parameterized Hypercomplex Convolutionscited by 26
  11. 2018Quaternion Convolutional Neural Networks for End-to-End Automatic Speech Recognitioncited by 21
  12. 2020On the use of Self-supervised Pre-trained Acoustic and Linguistic Features for Continuous Speech Emotion Recognitioncited by 19
13 more
  1. 2018Quaternion Convolutional Neural Networks for Detection and Localization of 3D Sound Eventscited by 18
  2. 2021Quaternion Generative Adversarial Networkscited by 17
  3. 2021Speech Emotion Recognition with Multiscale Area Attention and Data Augmentationcited by 12
  4. 2020A Quaternion-Valued Variational Autoencodercited by 11
  5. 2021Speech Emotion Recognition Using Quaternion Convolutional Neural Networkscited by 11
  6. 2018Adversarial Machine Learning And Speech Emotion Recognition: Utilizing Generative Adversarial Networks For Robustnesscited by 9
  7. 2017Emotion Recognition From Speech With Recurrent Neural Networkscited by 8
  8. 2022Dual Quaternion Ambisonics Array for Six-Degree-of-Freedom Acoustic Representationcited by 8
  9. 2017Capturing Long-term Temporal Dependencies with Convolutional Networks for Continuous Emotion Recognitioncited by 6
  10. 2020Quaternion Neural Networks for Multi-channel Distant Speech Recognitioncited by 5
  11. 2019Unsupervised Representation Learning with Future Observation Prediction for Speech Emotion Recognitioncited by 4
  12. 2021Hybrid Data Augmentation and Deep Attention-based Dilated Convolutional-Recurrent Neural Networks for Speech Emotion Recognitioncited by 3
  13. 2020Multi-Window Data Augmentation Approach for Speech Emotion Recognitioncited by 1

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 5.

7 papers.

  1. 2024Universal Approximation Theorem for Vector- and Hypercomplex-Valued Neural Networkscited by 11
  2. 2023MSAC: Multiple Speech Attribute Control Method for Reliable Speech Emotion Recognitioncited by 4
  3. 2023Active Learning Based Fine-Tuning Framework for Speech Emotion Recognitioncited by 2
  4. 2024PHemoNet: A Multimodal Network for Physiological Signalscited by 2
  5. 2024Demystifying the Hypercomplex: Inductive Biases in Hypercomplex Deep Learningcited by 1
  6. 2024Active Learning with Task Adaptation Pre-training for Speech Emotion Recognitioncited by 0
  7. 2024Multi-scale Quaternion CNN and BiGRU with Cross Self-attention Feature Fusion for Fault Diagnosis of Bearingcited by 0