mlsanityData to 14 Oct 2026

Exploring Self-Supervised Multi-view Contrastive Learning for Speech Emotion Recognition with Limited Annotations

arXiv 2406.07900 ↗First posted 12 Jun 2024cs.CL cs.AI cs.SDcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent advancements in Deep and Self-Supervised Learning (SSL) have led to substantial improvements in Speech Emotion Recognition (SER) performance, reaching unprecedented levels. However, obtaining sufficient amounts of accurately labeled data for training or fine-tuning the models remains a costly and challenging task. In this paper, we propose a multi-view SSL pre-training technique that can be applied to various representations of speech, including the ones generated by large speech models, to improve SER performance in scenarios where annotations are limited. Our experiments, based on wav2vec 2.0, spectral and paralinguistic features, demonstrate that the proposed framework boosts the SER performance, by up to 10% in Unweighted Average Recall, in settings with extremely sparse data annotations.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 16 of its 31 references that are in the map. First posted on arXiv 12 Jun 2024.

Builds on

16 of 31 references.

References · 16

  1. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  2. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  3. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  4. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  5. 2019Contrastive Multiview Codingcited by 1,301
  6. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,175
  7. 2018Insights on representational similarity in neural networks with canonical correlationcited by 324
  8. 2020Contrastive Learning of General-Purpose Audio Representationscited by 169
  9. 2021Emotion Recognition from Speech Using Wav2vec 2.0 Embeddingscited by 162
  10. 2021A Fine-tuned Wav2vec 2.0/HuBERT Benchmark For Speech Emotion Recognition, Speaker Verification and Spoken Language Understandingcited by 82
  11. 2021Multi-level Feature Learning for Contrastive Multi-view Clusteringcited by 73
  12. 2022Speech Emotion Recognition using Self-Supervised Featurescited by 53
4 more
  1. 2021Exploring Wav2vec 2.0 fine-tuning for improved speech emotion recognitioncited by 49
  2. 2022Exploration of A Self-Supervised Speech Model: A Study on Emotional Corporacited by 31
  3. 2023Speech Self-Supervised Representations Benchmarking: a Case for Larger Probing Headscited by 17
  4. 2022Contrastive Learning with Cross-Modal Knowledge Mining for Multimodal Human Activity Recognitioncited by 9

Built on by

Not cited yet.