# Towards Effective Negation Modeling in Joint Audio-Text Models for Music

arXiv: https://arxiv.org/abs/2601.13931 · HTML: https://mlsanity.com/paper/2601.13931/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 16 references that are in the map.
- First posted on arXiv 20 Jan 2026.

## Builds on (most cited first, 11 of 11)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [When and why vision-language models behave like bags-of-words, and what to do about it?](https://mlsanity.com/paper/2210.01936/) (2022)
- [AudioLDM: Text-to-Audio Generation with Latent Diffusion Models](https://mlsanity.com/paper/2301.12503/) (2023)
- [LP-MusicCaps: LLM-Based Pseudo Music Captioning](https://mlsanity.com/paper/2307.16372/) (2023)
- [Learn "No" to Say "Yes" Better: Improving Vision-Language Models via Negations](https://mlsanity.com/paper/2403.20312/) (2024)
- [Contrastive Audio-Language Learning for Music](https://mlsanity.com/paper/2208.12208/) (2022)
- [Diff-A-Riff: Musical Accompaniment Co-creation via Latent Diffusion Models](https://mlsanity.com/paper/2406.08384/) (2024)
- [Toward Universal Text-to-Music Retrieval](https://mlsanity.com/paper/2211.14558/) (2022)
- [Learn to Understand Negation in Video Retrieval](https://mlsanity.com/paper/2205.00132/) (2022)
- [I can listen but cannot read: An evaluation of two-tower multimodal systems for instrument recognition](https://mlsanity.com/paper/2407.18058/) (2024)
- [Evaluation of pretrained language models on music understanding](https://mlsanity.com/paper/2409.11449/) (2024)

## Built on by (most cited first, 1 of 1)

- [Extending Music Annotation Schemas: Zero-Shot Prediction or Few-Shot Adaptation?](https://mlsanity.com/paper/2610.06920/) (2026)

## Abstract (arXiv)

Joint audio-text models are widely used for music retrieval, yet they struggle with semantic phenomena such as negation. Negation is fundamental for distinguishing the absence (or presence) of musical elements (e.g., "with vocals" vs. "without vocals"), but current systems fail to represent this reliably. In this work, we investigate and mitigate this limitation by training CLAP models from scratch on the Million Song Dataset with LP-MusicCaps-MSD captions. We introduce negation through text augmentation and a dissimilarity-based contrastive loss, designed to explicitly separate original and negated captions in the joint embedding space. To evaluate progress, we propose two protocols that frame negation modeling as retrieval and binary classification tasks. Experiments demonstrate that both methods, individually and combined, improve negation handling while largely preserving retrieval performance.