# CLEP-DG: Contrastive Learning for Speech Emotion Domain Generalization via Soft Prompt Tuning

arXiv: https://arxiv.org/abs/2507.04048 · HTML: https://mlsanity.com/paper/2507.04048/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jul 2025.

## Built on by (most cited first, 7 of 7)

- [Emotion-Aligned Generation in Diffusion Text to Speech Models via Preference-Guided Optimization](https://mlsanity.com/paper/2509.25416/) (2025)
- [EMO-TTA: Improving Test-Time Adaptation of Audio-Language Models for Speech Emotion Recognition](https://mlsanity.com/paper/2509.25495/) (2025)
- [Adapting Where It Matters: Depth-Aware Adaptation for Efficient Multilingual Speech Recognition in Low-Resource Languages](https://mlsanity.com/paper/2602.01008/) (2026)
- [Plug-and-Play Emotion Graphs for Compositional Prompting in Zero-Shot Speech Emotion Recognition](https://mlsanity.com/paper/2509.25458/) (2025)
- [Generalizable Prompt Tuning for Audio-Language Models via Semantic Expansion](https://mlsanity.com/paper/2601.20867/) (2026)
- [Continual Adaptation for Pacific Indigenous Speech Recognition](https://mlsanity.com/paper/2603.06310/) (2026)
- [Acoustic Prompting via Stage-wise Modulation for Few-Shot Learning in Audio Language Models](https://mlsanity.com/paper/2606.15751/) (2026)

## Abstract (arXiv)

Speech Emotion Recognition (SER) is fundamental to affective computing and human-computer interaction, yet existing models struggle to generalize across diverse acoustic conditions. While Contrastive Language-Audio Pretraining (CLAP) provides strong multimodal alignment, it lacks dedicated mechanisms for capturing emotional cues, making it suboptimal for SER. To address this, we propose CLEP-DG, a framework that enhances CLAP's robustness in emotion recognition. First, we fine-tune CLAP to obtain CLEP, adapting it on large-scale emotional speech datasets to better encode emotion-relevant features. Then, we introduce Acoustic Context Prompt Tuning (ACPT), a text-driven augmentation strategy that optimizes learnable prompt vectors to model diverse acoustic environments without additional labeled audio. Finally, leveraging cross-modal transferability, we train a classifier on text-derived embeddings and apply it to the audio encoder during inference, mitigating domain shifts between textual supervision and audio-based emotion recognition. Experiments across five benchmark datasets show that CLEP-DG outperforms prior CLAP-based approaches, achieving state-of-the-art performance in both supervised and domain generalization settings.