# Multimodal Emotion Recognition from Raw Audio with Sinc-convolution

arXiv: https://arxiv.org/abs/2402.11954 · HTML: https://mlsanity.com/paper/2402.11954/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Feb 2024.

## Built on by (most cited first, 2 of 2)

- [GatedxLSTM: A Multimodal Affective Computing Approach for Emotion Recognition in Conversations](https://mlsanity.com/paper/2503.20919/) (2025)
- [Representation Learning with Parameterised Quantum Circuits for Advancing Speech Emotion Recognition](https://mlsanity.com/paper/2501.12050/) (2025)

## Abstract (arXiv)

Speech Emotion Recognition (SER) is still a complex task for computers with average recall rates usually about 70% on the most realistic datasets. Most SER systems use hand-crafted features extracted from audio signal such as energy, zero crossing rate, spectral information, prosodic, mel frequency cepstral coefficient (MFCC), and so on. More recently, using raw waveform for training neural network is becoming an emerging trend. This approach is advantageous as it eliminates the feature extraction pipeline. Learning from time-domain signal has shown good results for tasks such as speech recognition, speaker verification etc. In this paper, we utilize Sinc-convolution layer, which is an efficient architecture for preprocessing raw speech waveform for emotion recognition, to extract acoustic features from raw audio signals followed by a long short-term memory (LSTM). We also incorporate linguistic features and append a dialogical emotion decoding (DED) strategy. Our approach achieves a weighted accuracy of 85.1% in four class emotion on the Interactive Emotional Dyadic Motion Capture (IEMOCAP) dataset.