# Speech Emotion Recognition Using Quaternion Convolutional Neural Networks

arXiv: https://arxiv.org/abs/2111.00404 · HTML: https://mlsanity.com/paper/2111.00404/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 31 references that are in the map.
- First posted on arXiv 31 Oct 2021.

## Builds on (most cited first, 8 of 8)

- [DialogueRNN: An Attentive RNN for Emotion Detection in Conversations](https://mlsanity.com/paper/1811.00405/) (2018)
- [DialogueGCN: A Graph Convolutional Neural Network for Emotion Recognition in Conversation](https://mlsanity.com/paper/1908.11540/) (2019)
- [SEGAN: Speech Enhancement Generative Adversarial Network](https://mlsanity.com/paper/1703.09452/) (2017)
- [EESEN: End-to-End Speech Recognition using Deep RNN Models and WFST-based Decoding](https://mlsanity.com/paper/1507.08240/) (2015)
- [Quaternion Convolutional Neural Networks](https://mlsanity.com/paper/1903.00658/) (2019)
- [A Wavenet for Speech Denoising](https://mlsanity.com/paper/1706.07162/) (2017)
- [Quaternion Convolutional Neural Networks for End-to-End Automatic Speech Recognition](https://mlsanity.com/paper/1806.07789/) (2018)
- [Cross Lingual Cross Corpus Speech Emotion Recognition](https://mlsanity.com/paper/2003.07996/) (2020)

## Built on by (most cited first, 11 of 11)

- [Temporal Modeling Matters: A Novel Temporal Emotional Modeling Approach for Speech Emotion Recognition](https://mlsanity.com/paper/2211.08233/) (2022)
- [Learning Speech Emotion Representations in the Quaternion Domain](https://mlsanity.com/paper/2204.02385/) (2022)
- [Improving Speech Emotion Recognition Through Focus and Calibration Attention Mechanisms](https://mlsanity.com/paper/2208.10491/) (2022)
- [Quaternion Convolutional Neural Networks: Current Advances and Future Directions](https://mlsanity.com/paper/2307.08663/) (2023)
- [Time Series Compression using Quaternion Valued Neural Networks and Quaternion Backpropagation](https://mlsanity.com/paper/2403.11722/) (2024)
- [PHemoNet: A Multimodal Network for Physiological Signals](https://mlsanity.com/paper/2410.00010/) (2024)
- [deep learning of segment-level feature representation for speech emotion recognition in conversations](https://mlsanity.com/paper/2302.02419/) (2023)
- [Demystifying the Hypercomplex: Inductive Biases in Hypercomplex Deep Learning](https://mlsanity.com/paper/2405.07024/) (2024)
- [Improving Quaternion Neural Networks with Quaternionic Activation Functions](https://mlsanity.com/paper/2406.16481/) (2024)
- [Multi-scale Quaternion CNN and BiGRU with Cross Self-attention Feature Fusion for Fault Diagnosis of Bearing](https://mlsanity.com/paper/2405.16114/) (2024)
- [EmoHRNet: High-Resolution Neural Network Based Speech Emotion Recognition](https://mlsanity.com/paper/2510.06072/) (2025)

## Abstract (arXiv)

Although speech recognition has become a widespread technology, inferring emotion from speech signals still remains a challenge. To address this problem, this paper proposes a quaternion convolutional neural network (QCNN) based speech emotion recognition (SER) model in which Mel-spectrogram features of speech signals are encoded in an RGB quaternion domain. We show that our QCNN based SER model outperforms other real-valued methods in the Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS, 8-classes) dataset, achieving, to the best of our knowledge, state-of-the-art results. The QCNN also achieves comparable results with the state-of-the-art methods in the Interactive Emotional Dyadic Motion Capture (IEMOCAP 4-classes) and Berlin EMO-DB (7-classes) datasets. Specifically, the model achieves an accuracy of 77.87%, 70.46%, and 88.78% for the RAVDESS, IEMOCAP, and EMO-DB datasets, respectively. In addition, our results show that the quaternion unit structure is better able to encode internal dependencies to reduce its model size significantly compared to other methods.