mlsanityData to 14 Oct 2026

GEmo-CLAP: Gender-Attribute-Enhanced Contrastive Language-Audio Pretraining for Accurate Speech Emotion Recognition

arXiv 2306.07848 ↗First posted 13 Jun 2023cs.CL cs.MM cs.SDcited by 10

Not on the map. Why?

Abstract (arXiv)

Contrastive cross-modality pretraining has recently exhibited impressive success in diverse fields, whereas there is limited research on their merits in speech emotion recognition (SER). In this paper, we propose GEmo-CLAP, a kind of gender-attribute-enhanced contrastive language-audio pretraining (CLAP) method for SER. Specifically, we first construct an effective emotion CLAP (Emo-CLAP) for SER, using pre-trained text and audio encoders. Second, given the significance of gender information in SER, two novel multi-task learning based GEmo-CLAP (ML-GEmo-CLAP) and soft label based GEmo-CLAP (SL-GEmo-CLAP) models are further proposed to incorporate gender information of speech signals, forming more reasonable objectives. Experiments on IEMOCAP indicate that our proposed two GEmo-CLAPs consistently outperform Emo-CLAP with different pre-trained models. Remarkably, the proposed WavLM-based SL-GEmo-CLAP obtains the best WAR of 83.16%, which performs better than state-of-the-art SER methods.

Key facts. Cited by 10 papers in the map (data through 14 Oct 2026). Builds on 18 of its 28 references that are in the map. First posted on arXiv 13 Jun 2023.

Builds on

18 of 28 references.

References · 18

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,372
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  4. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  5. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  6. 2022data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Languagecited by 727
  7. 2022Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentationcited by 719
  8. 2021Emotion Recognition from Speech Using Wav2vec 2.0 Embeddingscited by 162
  9. 2022Speech Emotion Recognition using Self-Supervised Featurescited by 53
  10. 2021Exploring Wav2vec 2.0 fine-tuning for improved speech emotion recognitioncited by 49
  11. 2022Temporal Modeling Matters: A Novel Temporal Emotional Modeling Approach for Speech Emotion Recognitioncited by 19
  12. 2021Light-SERNet: A lightweight fully convolutional neural network for speech emotion recognitioncited by 14
6 more
  1. 2022Speaker Normalization for Self-supervised Speech Emotion Recognitioncited by 14
  2. 2021Key-Sparse Transformer for Multimodal Speech Emotion Recognitioncited by 10
  3. 2022Multi-level Fusion of Wav2vec 2.0 and BERT for Multimodal Emotion Recognitioncited by 9
  4. 2023Using Auxiliary Tasks In Multimodal Fusion Of Wav2vec 2.0 And BERT For Multimodal Emotion Recognitioncited by 9
  5. 2023Knowledge-aware Bayesian Co-attention for Multimodal Emotion Recognitioncited by 8
  6. 2022MMER: Multimodal Multi-task Learning for Speech Emotion Recognitioncited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 4; 2025: 5; 2026: 1.

10 papers.

  1. 2024Takin: A Cohort of Superior Quality Zero-shot Speech Generation Modelscited by 7
  2. 2025ReFlow-VC: Zero-shot Voice Conversion Based on Rectified Flow and Speaker Feature Optimizationcited by 4
  3. 2024Zero-Shot Voice Conversion via Content-Aware Timbre Ensemble and Conditional Flow Matchingcited by 3
  4. 2025GatedxLSTM: A Multimodal Affective Computing Approach for Emotion Recognition in Conversationscited by 3
  5. 2025ClapFM-EVC: High-Fidelity and Flexible Emotional Voice Conversion with Dual Control from Natural Language and Speechcited by 3
  6. 2024Takin-VC: Expressive Zero-Shot Voice Conversion via Adaptive Hybrid Content Encoding and Enhanced Timbre Modelingcited by 1
  7. 2025Balancing Task-invariant Interaction and Task-specific Adaptation for Unified Image Fusioncited by 1
  8. 2025EmotionRankCLAP: Bridging Natural Language Speaking Styles and Ordinal Speech Emotion via Rank-N-Contrastcited by 1
  9. 2024Can We Estimate Purchase Intention Based on Zero-shot Speech Emotion Recognition?cited by 0
  10. 2026Audio emotion recognition for atypical hearingcited by 0