# Two-stage Framework for Robust Speech Emotion Recognition Using Target Speaker Extraction in Human Speech Noise Conditions

arXiv: https://arxiv.org/abs/2409.19585 · HTML: https://mlsanity.com/paper/2409.19585/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 27 references that are in the map.
- First posted on arXiv 29 Sep 2024.

## Builds on (most cited first, 10 of 10)

- [SDR - half-baked or well done?](https://mlsanity.com/paper/1811.02508/) (2018)
- [LibriMix: An Open-Source Dataset for Generalizable Speech Separation](https://mlsanity.com/paper/2005.11262/) (2020)
- [Emotional Voice Conversion: Theory, Databases and ESD](https://mlsanity.com/paper/2105.14762/) (2021)
- [Exploring Wav2vec 2.0 fine-tuning for improved speech emotion recognition](https://mlsanity.com/paper/2110.06309/) (2021)
- [Speech Emotion Recognition with Co-Attention based Multi-level Acoustic Information](https://mlsanity.com/paper/2203.15326/) (2022)
- [Improving speaker discrimination of target speech extraction with time-domain SpeakerBeam](https://mlsanity.com/paper/2001.08378/) (2020)
- [Mingling or Misalignment? Temporal Shift for Speech Emotion Recognition with Pre-trained Representations](https://mlsanity.com/paper/2302.13277/) (2023)
- [A Comprehensive Study on the Effectiveness of ASR Representations for Noise-Robust Speech Emotion Recognition](https://mlsanity.com/paper/2311.07093/) (2023)
- [Multi-Level Knowledge Distillation for Speech Emotion Recognition in Noisy Conditions](https://mlsanity.com/paper/2312.13556/) (2023)
- [Analysis of impact of emotions on target speech extraction and speech separation](https://mlsanity.com/paper/2208.07091/) (2022)

## Built on by (most cited first, 2 of 2)

- [GIA-MIC: Multimodal Emotion Recognition with Gated Interactive Attention and Modality-Invariant Learning Constraints](https://mlsanity.com/paper/2506.00865/) (2025)
- [PARCO: Phoneme-Augmented Robust Contextual ASR via Contrastive Entity Disambiguation](https://mlsanity.com/paper/2509.04357/) (2025)

## Abstract (arXiv)

Developing a robust speech emotion recognition (SER) system in noisy conditions faces challenges posed by different noise properties. Most previous studies have not considered the impact of human speech noise, thus limiting the application scope of SER. In this paper, we propose a novel two-stage framework for the problem by cascading target speaker extraction (TSE) method and SER. We first train a TSE model to extract the speech of target speaker from a mixture. Then, in the second stage, we utilize the extracted speech for SER training. Additionally, we explore a joint training of TSE and SER models in the second stage. Our developed system achieves a 14.33% improvement in unweighted accuracy (UA) compared to a baseline without using TSE method, demonstrating the effectiveness of our framework in mitigating the impact of human speech noise. Moreover, we conduct experiments considering speaker gender, showing that our framework performs particularly well in different-gender mixture.