# Context and System Fusion in Post-ASR Emotion Recognition with Large Language Models

arXiv: https://arxiv.org/abs/2410.03312 · HTML: https://mlsanity.com/paper/2410.03312/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 18 references that are in the map.
- First posted on arXiv 4 Oct 2024.

## Builds on (most cited first, 11 of 11)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [Multi-modal Attention for Speech Emotion Recognition](https://mlsanity.com/paper/2009.04107/) (2020)
- [Fusing ASR Outputs in Joint Training for Speech Emotion Recognition](https://mlsanity.com/paper/2110.15684/) (2021)
- [Foundation Model Assisted Automatic Speech Emotion Recognition: Transcribing, Annotating, and Augmenting](https://mlsanity.com/paper/2309.08108/) (2023)
- [Speech Emotion Recognition with ASR Transcripts: A Comprehensive Study on Word Error Rate and Fusion Techniques](https://mlsanity.com/paper/2406.08353/) (2024)
- [Large Language Model Based Generative Error Correction: A Challenge and Baselines for Speech Recognition, Speaker Tagging, and Emotion Recognition](https://mlsanity.com/paper/2409.09785/) (2024)
- [Crossmodal ASR Error Correction with Discrete Speech Units](https://mlsanity.com/paper/2405.16677/) (2024)
- [LanSER: Language-Model Supported Speech Emotion Recognition](https://mlsanity.com/paper/2309.03978/) (2023)
- [The Highs and Lows of Simple Lexical Domain Adaptation Approaches for Neural Machine Translation](https://mlsanity.com/paper/2101.00421/) (2021)

## Built on by (most cited first, 1 of 1)

- [A Mixture-of-Experts Model for Multimodal Emotion Recognition in Conversations](https://mlsanity.com/paper/2602.23300/) (2026)

## Abstract (arXiv)

Large language models (LLMs) have started to play a vital role in modelling speech and text. To explore the best use of context and multiple systems' outputs for post-ASR speech emotion prediction, we study LLM prompting on a recent task named GenSEC. Our techniques include ASR transcript ranking, variable conversation context, and system output fusion. We show that the conversation context has diminishing returns and the metric used to select the transcript for prediction is crucial. Finally, our best submission surpasses the provided baseline by 20% in absolute accuracy.