# Speech Emotion Recognition with Dual-Sequence LSTM Architecture

arXiv: https://arxiv.org/abs/1910.08874 · HTML: https://mlsanity.com/paper/1910.08874/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Oct 2019.

## Built on by (most cited first, 23 of 23)

- [Speech Emotion Recognition using Self-Supervised Features](https://mlsanity.com/paper/2202.03896/) (2022)
- [Pruning Deep Neural Networks from a Sparsity Perspective](https://mlsanity.com/paper/2302.05601/) (2023)
- [DER-GCN: Dialogue and Event Relation-Aware Graph Convolutional Neural Network for Multimodal Dialogue Emotion Recognition](https://mlsanity.com/paper/2312.10579/) (2023)
- [Temporal Modeling Matters: A Novel Temporal Emotional Modeling Approach for Speech Emotion Recognition](https://mlsanity.com/paper/2211.08233/) (2022)
- [Speaker Normalization for Self-supervised Speech Emotion Recognition](https://mlsanity.com/paper/2202.01252/) (2022)
- [A cross-modal fusion network based on self-attention and residual structure for multimodal emotion recognition](https://mlsanity.com/paper/2111.02172/) (2021)
- [CapsField: Light Field-based Face and Expression Recognition in the Wild using Capsule Routing](https://mlsanity.com/paper/2101.03503/) (2021)
- [MMER: Multimodal Multi-task Learning for Speech Emotion Recognition](https://mlsanity.com/paper/2203.16794/) (2022)
- [The Role of Phonetic Units in Speech Emotion Recognition](https://mlsanity.com/paper/2108.01132/) (2021)
- [Robust Federated Learning Against Adversarial Attacks for Speech Emotion Recognition](https://mlsanity.com/paper/2203.04696/) (2022)
- [Neural Architecture Search for Speech Emotion Recognition](https://mlsanity.com/paper/2203.16928/) (2022)
- [Multi-Perspective LSTM for Joint Visual Representation Learning](https://mlsanity.com/paper/2105.02802/) (2021)
- [Improving Speech Emotion Recognition Through Focus and Calibration Attention Mechanisms](https://mlsanity.com/paper/2208.10491/) (2022)
- [Speech Emotion Recognition via an Attentive Time-Frequency Neural Network](https://mlsanity.com/paper/2210.12430/) (2022)
- [Exploring Emotion Expression Recognition in Older Adults Interacting with a Virtual Coach](https://mlsanity.com/paper/2311.05567/) (2023)
- [TACFN: Transformer-based Adaptive Cross-modal Fusion Network for Multimodal Emotion Recognition](https://mlsanity.com/paper/2505.06536/) (2025)
- [A Study on the Ambiguity in Human Annotation of German Oral History Interviews for Perceived Emotion Recognition and Sentiment Analysis](https://mlsanity.com/paper/2201.06868/) (2022)
- [What Does it Take to Generalize SER Model Across Datasets? A Comprehensive Benchmark](https://mlsanity.com/paper/2406.09933/) (2024)
- [Towards a Common Speech Analysis Engine](https://mlsanity.com/paper/2203.00613/) (2022)
- [Are Mamba-based Audio Foundation Models the Best Fit for Non-Verbal Emotion Recognition?](https://mlsanity.com/paper/2506.02258/) (2025)
- [EmoHRNet: High-Resolution Neural Network Based Speech Emotion Recognition](https://mlsanity.com/paper/2510.06072/) (2025)
- [Expert-Guided Multimodal Fusion for Unified Emotion and Sentiment Analysis](https://mlsanity.com/paper/2601.07565/) (2026)
- [Cross-Modal Emotion Understanding: A Transformer-GAT Approach for Dialogue Emotion Recognition](https://mlsanity.com/paper/2609.29556/) (2026)

## Abstract (arXiv)

Speech Emotion Recognition (SER) has emerged as a critical component of the next generation human-machine interfacing technologies. In this work, we propose a new dual-level model that predicts emotions based on both MFCC features and mel-spectrograms produced from raw audio signals. Each utterance is preprocessed into MFCC features and two mel-spectrograms at different time-frequency resolutions. A standard LSTM processes the MFCC features, while a novel LSTM architecture, denoted as Dual-Sequence LSTM (DS-LSTM), processes the two mel-spectrograms simultaneously. The outputs are later averaged to produce a final classification of the utterance. Our proposed model achieves, on average, a weighted accuracy of 72.7% and an unweighted accuracy of 73.3%---a 6% improvement over current state-of-the-art unimodal models---and is comparable with multimodal models that leverage textual information as well as audio signals.