mlsanityData to 7 Oct 2026

A Comprehensive Study on the Effectiveness of ASR Representations for Noise-Robust Speech Emotion Recognition

arXiv 2311.07093 ↗First posted 13 Nov 2023cs.SD cs.CL eess.AScited by 4

Not on the map. Why?

Abstract (arXiv)

This paper proposes an efficient attempt to noisy speech emotion recognition (NSER). Conventional NSER approaches have proven effective in mitigating the impact of artificial noise sources, such as white Gaussian noise, but are limited to non-stationary noises in real-world environments due to their complexity and uncertainty. To overcome this limitation, we introduce a new method for NSER by adopting the automatic speech recognition (ASR) model as a noise-robust feature extractor to eliminate non-vocal information in noisy speech. We first obtain intermediate layer information from the ASR model as a feature representation for emotional speech and then apply this representation for the downstream NSER task. Our experimental results show that 1) the proposed method achieves better NSER performance compared with the conventional noise reduction method, 2) outperforms self-supervised learning approaches, and 3) even outperforms text-based approaches using ASR transcription or the ground truth transcription of noisy speech.

Key facts. Cited by 4 papers in the map (data through 7 Oct 2026). Builds on 22 of its 81 references that are in the map. First posted on arXiv 13 Nov 2023.

Builds on

22 of 81 references.

References · 22

  1. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,016
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  4. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  5. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  6. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  7. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  8. 2018MELD: A Multimodal Multi-Party Dataset for Emotion Recognition in Conversationscited by 636
  9. 2018Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separationcited by 400
  10. 2022Dawn of the transformer era in speech emotion recognition: closing the valence gapcited by 155
  11. 2020COSMIC: COmmonSense knowledge for eMotion Identification in Conversationscited by 135
  12. 2020DCCRN: Deep Complex Convolution Recurrent Network for Phase-Aware Speech Enhancementcited by 133
10 more
  1. 2023Whisper-AT: Noise-Robust Automatic Speech Recognizers are Also Strong General Audio Event Taggerscited by 72
  2. 2018Multimodal Speech Emotion Recognition Using Audio and Textcited by 55
  3. 2022SpeechLM: Enhanced Speech Pre-Training with Unpaired Textual Datacited by 49
  4. 2020Emotion recognition by fusing time synchronous and time asynchronous representationscited by 26
  5. 2019Speech Sentiment Analysis via Pre-trained Features from End-to-end ASR Modelscited by 21
  6. 2021Fusing ASR Outputs in Joint Training for Speech Emotion Recognitioncited by 21
  7. 2023Foundation Model Assisted Automatic Speech Emotion Recognition: Transcribing, Annotating, and Augmentingcited by 17
  8. 2020Learning Fine-Grained Cross Modality Excitement for Speech Emotion Recognitioncited by 10
  9. 2021Leveraging Pre-trained Language Model for Speech Sentiment Analysiscited by 8
  10. 2021End-to-End Speech Emotion Recognition: Challenges of Real-Life Emergency Call Centers Data Recordingscited by 8

Built on by

Cited by year

Citing papers per year: 2024: 3; 2025: 1.

4 papers.

  1. 2024MF-AED-AEC: Speech Emotion Recognition by Leveraging Multimodal Fusion, Asr Error Detection, and Asr Error Correctioncited by 8
  2. 2024Adapting Whisper for Code-Switching through Encoding Refining and Language-Aware Decodingcited by 7
  3. 2024Two-stage Framework for Robust Speech Emotion Recognition Using Target Speaker Extraction in Human Speech Noise Conditionscited by 2
  4. 2025Combining Deterministic Enhanced Conditions with Dual-Streaming Encoding for Diffusion-Based Speech Enhancementcited by 0