# Do Joint Language-Audio Embeddings Encode Perceptual Timbre Semantics?

arXiv: https://arxiv.org/abs/2510.14249 · HTML: https://mlsanity.com/paper/2510.14249/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 18 references that are in the map.
- First posted on arXiv 16 Oct 2025.

## Builds on (most cited first, 9 of 9)

- [Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentation](https://mlsanity.com/paper/2211.06687/) (2022)
- [AudioLDM: Text-to-Audio Generation with Latent Diffusion Models](https://mlsanity.com/paper/2301.12503/) (2023)
- [MuQ: Self-Supervised Music Representation Learning with Mel Residual Vector Quantization](https://mlsanity.com/paper/2501.01108/) (2025)
- [Natural Language Supervision for General-Purpose Audio Representations](https://mlsanity.com/paper/2309.05767/) (2023)
- [Audio Retrieval with WavText5K and CLAP Training](https://mlsanity.com/paper/2209.14275/) (2022)
- [EnCLAP: Combining Neural Audio Codec and Audio-Text Joint Embedding for Automated Audio Captioning](https://mlsanity.com/paper/2401.17690/) (2024)
- [FLAM: Frame-Wise Language-Audio Modeling](https://mlsanity.com/paper/2505.05335/) (2025)
- [Text2FX: Harnessing CLAP Embeddings for Text-Guided Audio Effects](https://mlsanity.com/paper/2409.18847/) (2024)
- [Assessing the Alignment of Audio Representations with Timbre Similarity Ratings](https://mlsanity.com/paper/2507.07764/) (2025)

## Built on by (most cited first, 1 of 1)

- [Probing Low-Level Acoustic Attribute Encoding in CLAP Audio Embeddings](https://mlsanity.com/paper/2607.03806/) (2026)

## Abstract (arXiv)

Understanding and modeling the relationship between language and sound are essential for applications such as music information retrieval, text-guided music generation, and audio captioning. Central to these tasks are joint language-audio embedding spaces, which map textual descriptions and auditory content into a shared representation. Although multimodal embedding models such as MS-CLAP, LAION-CLAP, MuQ-MuLan, and OpenFLAM have shown strong performance in language-audio alignment, their correspondence to human perception of timbre, a multifaceted attribute encompassing qualities such as brightness, roughness, and warmth, remains under-explored. In this paper, we evaluate these joint language-audio embedding models in terms of their ability to capture perceptual timbre semantics. Across two complementary experiments, we find that LAION-CLAP shows relatively strong and consistent alignment with human-perceived timbre semantics across both instrumental sounds and descriptor-conditioned audio effects. At the same time, the overall strength of this alignment remains limited, suggesting that current joint language-audio embeddings capture perceptual timbre semantics only partially. We also observe that, overall, reverb-induced timbre semantics are more consistently encoded than equalization-induced timbre semantics.