# TRNet: Two-level Refinement Network leveraging Speech Enhancement for Noise Robust Speech Emotion Recognition

arXiv: https://arxiv.org/abs/2404.12979 · HTML: https://mlsanity.com/paper/2404.12979/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 15 of its 30 references that are in the map.
- First posted on arXiv 19 Apr 2024.

## Builds on (most cited first, 15 of 15)

- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [MUSAN: A Music, Speech, and Noise Corpus](https://mlsanity.com/paper/1510.08484/) (2015)
- [Supervised Speech Separation Based on Deep Learning: An Overview](https://mlsanity.com/paper/1708.07524/) (2017)
- [DCCRN: Deep Complex Convolution Recurrent Network for Phase-Aware Speech Enhancement](https://mlsanity.com/paper/2008.00264/) (2020)
- [CMGAN: Conformer-based Metric GAN for Speech Enhancement](https://mlsanity.com/paper/2203.15149/) (2022)
- [DPT-FSNet: Dual-path Transformer Based Full-band and Sub-band Fusion Network for Speech Enhancement](https://mlsanity.com/paper/2104.13002/) (2021)
- [ICASSP 2021 Deep Noise Suppression Challenge](https://mlsanity.com/paper/2009.06122/) (2020)
- [CMGAN: Conformer-Based Metric-GAN for Monaural Speech Enhancement](https://mlsanity.com/paper/2209.11112/) (2022)
- [ESPnet-se: end-to-end speech enhancement and separation toolkit designed for asr integration](https://mlsanity.com/paper/2011.03706/) (2020)
- [Improving noise robust automatic speech recognition with single-channel time-domain enhancement network](https://mlsanity.com/paper/2003.03998/) (2020)
- [Temporal Modeling Matters: A Novel Temporal Emotional Modeling Approach for Speech Emotion Recognition](https://mlsanity.com/paper/2211.08233/) (2022)
- [On the Robustness of Speech Emotion Recognition for Human-Robot Interaction with Deep Neural Networks](https://mlsanity.com/paper/1804.02173/) (2018)
- [Noise robust speech emotion recognition with signal-to-noise ratio adapting speech enhancement](https://mlsanity.com/paper/2309.01164/) (2023)
- [SNRi Target Training for Joint Speech Enhancement and Recognition](https://mlsanity.com/paper/2111.00764/) (2021)
- [CTA-RNN: Channel and Temporal-wise Attention RNN Leveraging Pre-trained ASR Embeddings for Speech Emotion Recognition](https://mlsanity.com/paper/2203.17023/) (2022)

## Abstract (arXiv)

One persistent challenge in Speech Emotion Recognition (SER) is the ubiquitous environmental noise, which frequently results in deteriorating SER performance in practice. In this paper, we introduce a Two-level Refinement Network, dubbed TRNet, to address this challenge. Specifically, a pre-trained speech enhancement module is employed for front-end noise reduction and noise level estimation. Later, we utilize clean speech spectrograms and their corresponding deep representations as reference signals to refine the spectrogram distortion and representation shift of enhanced speech during model training. Experimental results validate that the proposed TRNet substantially promotes the robustness of the proposed system in both matched and unmatched noisy environments, without compromising its performance in noise-free environments.