# ParaCLAP -- Towards a general language-audio model for computational paralinguistic tasks

arXiv: https://arxiv.org/abs/2406.07203 · HTML: https://mlsanity.com/paper/2406.07203/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jun 2024.

## Built on by (most cited first, 10 of 10)

- [Enhancing Emotional Text-to-Speech Controllability with Natural Language Guidance through Contrastive Learning and Diffusion Models](https://mlsanity.com/paper/2409.06451/) (2024)
- [GatedxLSTM: A Multimodal Affective Computing Approach for Emotion Recognition in Conversations](https://mlsanity.com/paper/2503.20919/) (2025)
- [EmotionRankCLAP: Bridging Natural Language Speaking Styles and Ordinal Speech Emotion via Rank-N-Contrast](https://mlsanity.com/paper/2505.23732/) (2025)
- [MELT: Towards Automated Multimodal Emotion Data Annotation by Leveraging LLM Embedded Knowledge](https://mlsanity.com/paper/2505.24493/) (2025)
- [ParaMETA: Towards Learning Disentangled Paralinguistic Speaking Styles Representations from Speech](https://mlsanity.com/paper/2601.12289/) (2026)
- [ParaSpeechCLAP: A Dual-Encoder Speech-Text Model for Rich Stylistic Language-Audio Pretraining](https://mlsanity.com/paper/2603.28737/) (2026)
- [The Sound of Absence: Audio-Language Embedding Models Struggle with Negation](https://mlsanity.com/paper/2607.12290/) (2026)
- [Prompt Amplification and Zero-Shot Late Fusion in Audio-Language Models for Speech Emotion Recognition](https://mlsanity.com/paper/2603.23057/) (2026)
- [DialBGM: A Benchmark for Background Music Recommendation from Everyday Multi-Turn Dialogues](https://mlsanity.com/paper/2604.07895/) (2026)
- [Acoustic Cue Alignment in Audio Language Models for Speech Emotion Recognition](https://mlsanity.com/paper/2606.07309/) (2026)

## Abstract (arXiv)

Contrastive language-audio pretraining (CLAP) has recently emerged as a method for making audio analysis more generalisable. Specifically, CLAP-style models are able to `answer' a diverse set of language queries, extending the capabilities of audio models beyond a closed set of labels. However, CLAP relies on a large set of (audio, query) pairs for pretraining. While such sets are available for general audio tasks, like captioning or sound event detection, there are no datasets with matched audio and text queries for computational paralinguistic (CP) tasks. As a result, the community relies on generic CLAP models trained for general audio with limited success. In the present study, we explore training considerations for ParaCLAP, a CLAP-style model suited to CP, including a novel process for creating audio-language queries. We demonstrate its effectiveness on a set of computational paralinguistic tasks, where it is shown to surpass the performance of open-source state-of-the-art models.