# CrossMuSim: A Cross-Modal Framework for Music Similarity Retrieval with LLM-Powered Text Description Sourcing and Mining

arXiv: https://arxiv.org/abs/2503.23128 · HTML: https://mlsanity.com/paper/2503.23128/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 23 references that are in the map.
- First posted on arXiv 29 Mar 2025.

## Builds on (most cited first, 14 of 14)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation](https://mlsanity.com/paper/2004.09813/) (2020)
- [MusicLM: Generating Music From Text](https://mlsanity.com/paper/2301.11325/) (2023)
- [MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Training](https://mlsanity.com/paper/2306.00107/) (2023)
- [Fast Timing-Conditioned Latent Audio Diffusion](https://mlsanity.com/paper/2402.04825/) (2024)
- [LP-MusicCaps: LLM-Based Pseudo Music Captioning](https://mlsanity.com/paper/2307.16372/) (2023)
- [Contrastive Learning of Musical Representations](https://mlsanity.com/paper/2103.09410/) (2021)
- [Contrastive Audio-Language Learning for Music](https://mlsanity.com/paper/2208.12208/) (2022)
- [Semi-Supervised Music Tagging Transformer](https://mlsanity.com/paper/2111.13457/) (2021)
- [MuChin: A Chinese Colloquial Description Benchmark for Evaluating Language Models in the Field of Music](https://mlsanity.com/paper/2402.09871/) (2024)
- [CLaMP 2: Multimodal Music Information Retrieval Across 101 Languages Using Large Language Models](https://mlsanity.com/paper/2410.13267/) (2024)
- [Representation Learning of Music Using Artist Labels](https://mlsanity.com/paper/1710.06648/) (2017)
- [Augment, Drop & Swap: Improving Diversity in LLM Captions for Efficient Music-Text Representation Learning](https://mlsanity.com/paper/2409.11498/) (2024)
- [Musical Audio Similarity with Self-supervised Convolutional Neural Networks](https://mlsanity.com/paper/2202.02112/) (2022)

## Built on by (most cited first, 1 of 1)

- [MUVR: A Multi-Modal Untrimmed Video Retrieval Benchmark with Multi-Level Visual Correspondence](https://mlsanity.com/paper/2510.21406/) (2025)

## Abstract (arXiv)

Music similarity retrieval is fundamental for managing and exploring relevant content from large collections in streaming platforms. This paper presents a novel cross-modal contrastive learning framework that leverages the open-ended nature of text descriptions to guide music similarity modeling, addressing the limitations of traditional uni-modal approaches in capturing complex musical relationships. To overcome the scarcity of high-quality text-music paired data, this paper introduces a dual-source data acquisition approach combining online scraping and LLM-based prompting, where carefully designed prompts leverage LLMs' comprehensive music knowledge to generate contextually rich descriptions. Exten1sive experiments demonstrate that the proposed framework achieves significant performance improvements over existing benchmarks through objective metrics, subjective evaluations, and real-world A/B testing on the Huawei Music streaming platform.