# On the Robustness of Speech Emotion Recognition for Human-Robot Interaction with Deep Neural Networks

arXiv: https://arxiv.org/abs/1804.02173 · HTML: https://mlsanity.com/paper/1804.02173/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Apr 2018.

## Built on by (most cited first, 8 of 8)

- [Robustness Analysis of Video-Language Models Against Visual and Language Perturbations](https://mlsanity.com/paper/2207.02159/) (2022)
- [Leveraging Speech PTM, Text LLM, and Emotional TTS for Speech Emotion Recognition](https://mlsanity.com/paper/2309.10294/) (2023)
- [A Preliminary Study on Augmenting Speech Emotion Recognition using a Diffusion Model](https://mlsanity.com/paper/2305.11413/) (2023)
- [Beyond Text: Utilizing Vocal Cues to Improve Decision Making in LLMs for Robot Navigation Tasks](https://mlsanity.com/paper/2402.03494/) (2024)
- [Improving Speech Emotion Recognition with Unsupervised Speaking Style Transfer](https://mlsanity.com/paper/2211.08843/) (2022)
- [Describe Where You Are: Improving Noise-Robustness for Speech Emotion Recognition with Text Description of the Environment](https://mlsanity.com/paper/2407.17716/) (2024)
- [On the Efficacy and Noise-Robustness of Jointly Learned Speech Emotion and Automatic Speech Recognition](https://mlsanity.com/paper/2305.12540/) (2023)
- [TRNet: Two-level Refinement Network leveraging Speech Enhancement for Noise Robust Speech Emotion Recognition](https://mlsanity.com/paper/2404.12979/) (2024)

## Abstract (arXiv)

Speech emotion recognition (SER) is an important aspect of effective human-robot collaboration and received a lot of attention from the research community. For example, many neural network-based architectures were proposed recently and pushed the performance to a new level. However, the applicability of such neural SER models trained only on in-domain data to noisy conditions is currently under-researched. In this work, we evaluate the robustness of state-of-the-art neural acoustic emotion recognition models in human-robot interaction scenarios. We hypothesize that a robot's ego noise, room conditions, and various acoustic events that can occur in a home environment can significantly affect the performance of a model. We conduct several experiments on the iCub robot platform and propose several novel ways to reduce the gap between the model's performance during training and testing in real-world conditions. Furthermore, we observe large improvements in the model performance on the robot and demonstrate the necessity of introducing several data augmentation techniques like overlaying background noise and loudness variations to improve the robustness of the neural approaches.