# An Empirical Study and Improvement for Speech Emotion Recognition

arXiv: https://arxiv.org/abs/2304.03899 · HTML: https://mlsanity.com/paper/2304.03899/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 21 references that are in the map.
- First posted on arXiv 8 Apr 2023.

## Builds on (most cited first, 9 of 9)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [MM-DFN: Multimodal Dynamic Fusion Network for Emotion Recognition in Conversations](https://mlsanity.com/paper/2203.02385/) (2022)
- [HiGRU: Hierarchical Gated Recurrent Units for Utterance-level Emotion Recognition](https://mlsanity.com/paper/1904.04446/) (2019)
- [Learning Alignment for Multimodal Emotion Recognition from Speech](https://mlsanity.com/paper/1909.05645/) (2019)
- [Speech Emotion Recognition Using Multi-hop Attention Mechanism](https://mlsanity.com/paper/1904.10788/) (2019)
- [Improving Review Representations with User Attention and Product Attention for Sentiment Classification](https://mlsanity.com/paper/1801.07861/) (2018)
- [Conversational Emotion Analysis via Attention Mechanisms](https://mlsanity.com/paper/1910.11263/) (2019)
- [Attentive Modality Hopping Mechanism for Speech Emotion Recognition](https://mlsanity.com/paper/1912.00846/) (2019)
- [Multimodal Speech Emotion Recognition using Cross Attention with Aligned Audio and Text](https://mlsanity.com/paper/2207.12895/) (2022)

## Abstract (arXiv)

Multimodal speech emotion recognition aims to detect speakers' emotions from audio and text. Prior works mainly focus on exploiting advanced networks to model and fuse different modality information to facilitate performance, while neglecting the effect of different fusion strategies on emotion recognition. In this work, we consider a simple yet important problem: how to fuse audio and text modality information is more helpful for this multimodal task. Further, we propose a multimodal emotion recognition model improved by perspective loss. Empirical results show our method obtained new state-of-the-art results on the IEMOCAP dataset. The in-depth analysis explains why the improved model can achieve improvements and outperforms baselines.