mlsanityData to 7 Oct 2026

Are Paralinguistic Representations all that is needed for Speech Emotion Recognition?

arXiv 2402.01579 ↗First posted 2 Feb 2024eess.AS cs.CL cs.SDcited by 1

Not on the map. Why?

Abstract (arXiv)

Availability of representations from pre-trained models (PTMs) have facilitated substantial progress in speech emotion recognition (SER). Particularly, representations from PTM trained for paralinguistic speech processing have shown state-of-the-art (SOTA) performance for SER. However, such paralinguistic PTM representations haven't been evaluated for SER in linguistic environments other than English. Also, paralinguistic PTM representations haven't been investigated in benchmarks such as SUPERB, EMO-SUPERB, ML-SUPERB for SER. This makes it difficult to access the efficacy of paralinguistic PTM representations for SER in multiple languages. To fill this gap, we perform a comprehensive comparative study of five SOTA PTM representations. Our results shows that paralinguistic PTM (TRILLsson) representations performs the best and this performance can be attributed to its effectiveness in capturing pitch, tone and other speech characteristics more effectively than other PTM representations.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 17 of its 31 references that are in the map. First posted on arXiv 2 Feb 2024.

Builds on

17 of 31 references.

References · 17

  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  3. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,609
  4. 2021SUPERB: Speech processing Universal PERformance Benchmarkcited by 652
  5. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 589
  6. 2020Unsupervised Cross-lingual Representation Learning for Speech Recognitioncited by 496
  7. 2021Emotion Recognition from Speech Using Wav2vec 2.0 Embeddingscited by 158
  8. 2023ML-SUPERB: Multilingual Speech Universal PERformance Benchmarkcited by 70
  9. 2022Speech Emotion Recognition using Self-Supervised Featurescited by 52
  10. 2021Universal Paralinguistic Speech Representations Using Self-Supervised Conformerscited by 32
  11. 2020x-vectors meet emotions: A study on dependencies between emotion and speaker recognitioncited by 31
  12. 2018Cross Lingual Speech Emotion Recognition: Urdu vs. Western Languagescited by 25
5 more
  1. 2021SERAB: A multi-lingual benchmark for speech emotion recognitioncited by 22
  2. 2022TRILLsson: Distilled Universal Paralinguistic Speech Representationscited by 16
  3. 2023Transforming the Embeddings: A Lightweight Technique for Speech Emotion Recognition Taskscited by 4
  4. 2023Decoding Emotions: A comprehensive Multilingual Study of Speech Models for Speech Emotion Recognitioncited by 4
  5. 2023Learning Multilingual Expressive Speech Representation for Prosody Prediction without Parallel Datacited by 3

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025PARROT: Synergizing Mamba and Attention-based SSL Pre-Trained Models via Parallel Branch Hadamard Optimal Transport for Speech Emotion Recognitioncited by 0