# Improving Speech-based Emotion Recognition with Contextual Utterance Analysis and LLMs

arXiv: https://arxiv.org/abs/2410.20334 · HTML: https://mlsanity.com/paper/2410.20334/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 15 references that are in the map.
- First posted on arXiv 27 Oct 2024.

## Builds on (most cited first, 10 of 10)

- [Listen, Think, and Understand](https://mlsanity.com/paper/2305.10790/) (2023)
- [Topic Modeling Based Multi-modal Depression Detection](https://mlsanity.com/paper/1803.10384/) (2018)
- [Fusing ASR Outputs in Joint Training for Speech Emotion Recognition](https://mlsanity.com/paper/2110.15684/) (2021)
- [Foundation Model Assisted Automatic Speech Emotion Recognition: Transcribing, Annotating, and Augmenting](https://mlsanity.com/paper/2309.08108/) (2023)
- [Speech Emotion Recognition with ASR Transcripts: A Comprehensive Study on Word Error Rate and Fusion Techniques](https://mlsanity.com/paper/2406.08353/) (2024)
- [Language and Mental Health: Measures of Emotion Dynamics from Text as Linguistic Biosocial Markers](https://mlsanity.com/paper/2310.17369/) (2023)
- [Large Language Model Based Generative Error Correction: A Challenge and Baselines for Speech Recognition, Speaker Tagging, and Emotion Recognition](https://mlsanity.com/paper/2409.09785/) (2024)
- [Can Large Language Models Aid in Annotating Speech Emotional Data? Uncovering New Frontiers](https://mlsanity.com/paper/2307.06090/) (2023)
- [On Prompt Sensitivity of ChatGPT in Affective Computing](https://mlsanity.com/paper/2403.14006/) (2024)
- [LanSER: Language-Model Supported Speech Emotion Recognition](https://mlsanity.com/paper/2309.03978/) (2023)

## Built on by (most cited first, 1 of 1)

- [A Mixture-of-Experts Model for Multimodal Emotion Recognition in Conversations](https://mlsanity.com/paper/2602.23300/) (2026)

## Abstract (arXiv)

Speech Emotion Recognition (SER) focuses on identifying emotional states from spoken language. The 2024 IEEE SLT-GenSEC Challenge on Post Automatic Speech Recognition (ASR) Emotion Recognition tasks participants to explore the capabilities of large language models (LLMs) for emotion recognition using only text data. We propose a novel approach that first refines all available transcriptions to ensure data reliability. We then segment each complete conversation into smaller dialogues and use these dialogues as context to predict the emotion of the target utterance within the dialogue. Finally, we investigated different context lengths and prompting techniques to improve prediction accuracy. Our best submission exceeded the baseline by 20% in unweighted accuracy, achieving the best performance in the challenge. All our experiments' codes, prediction results, and log files are publicly available.