# On the Contribution of Lexical Features to Speech Emotion Recognition

arXiv: https://arxiv.org/abs/2509.05634 · HTML: https://mlsanity.com/paper/2509.05634/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 23 references that are in the map.
- First posted on arXiv 6 Sep 2025.

## Builds on (most cited first, 18 of 18)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Unsupervised Cross-lingual Representation Learning at Scale](https://mlsanity.com/paper/1911.02116/) (2019)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [DeBERTa: Decoding-enhanced BERT with Disentangled Attention](https://mlsanity.com/paper/2006.03654/) (2020)
- [MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversations](https://mlsanity.com/paper/1810.02508/) (2018)
- [Seamless: Multilingual Expressive and Streaming Speech Translation](https://mlsanity.com/paper/2312.05187/) (2023)
- [Real Time Speech Enhancement in the Waveform Domain](https://mlsanity.com/paper/2006.12847/) (2020)
- [SpeechFormer: A Hierarchical Efficient Framework Incorporating the Characteristics of Speech](https://mlsanity.com/paper/2203.03812/) (2022)
- [SpeechFormer++: A Hierarchical Efficient Framework for Paralinguistic Speech Processing](https://mlsanity.com/paper/2302.14638/) (2023)
- [DST: Deformable Speech Transformer for Emotion Recognition](https://mlsanity.com/paper/2302.13729/) (2023)
- [Temporal-Frequency State Space Duality: An Efficient Paradigm for Speech Emotion Recognition](https://mlsanity.com/paper/2412.16904/) (2024)
- [DWFormer: Dynamic Window transFormer for Speech Emotion Recognition](https://mlsanity.com/paper/2303.01694/) (2023)
- [cross-modal fusion techniques for utterance-level emotion recognition from text and speech](https://mlsanity.com/paper/2302.02447/) (2023)
- [A Change of Heart: Improving Speech Emotion Recognition through Speech-to-Text Modality Conversion](https://mlsanity.com/paper/2307.11584/) (2023)
- [TADA: Training-free Attribution and Out-of-Domain Detection of Audio Deepfakes](https://mlsanity.com/paper/2506.05802/) (2025)
- [Unmasking real-world audio deepfakes: A data-centric approach](https://mlsanity.com/paper/2506.09606/) (2025)
- [An analysis of large speech models-based representations for speech emotion recognition](https://mlsanity.com/paper/2311.00394/) (2023)

## Abstract (arXiv)

Although paralinguistic cues are often considered the primary drivers of speech emotion recognition (SER), we investigate the role of lexical content extracted from speech and show that it can achieve competitive and in some cases higher performance compared to acoustic models. On the MELD dataset, our lexical-based approach obtains a weighted F1-score (WF1) of 51.5%, compared to 49.3% for an acoustic-only pipeline with a larger parameter count. Furthermore, we analyze different self-supervised (SSL) speech and text representations, conduct a layer-wise study of transformer-based encoders, and evaluate the effect of audio denoising.