# Toward Universal Text-to-Music Retrieval

arXiv: https://arxiv.org/abs/2211.14558 · HTML: https://mlsanity.com/paper/2211.14558/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Nov 2022.

## Built on by (most cited first, 18 of 18)

- [AudioLDM 2: Learning Holistic Audio Generation with Self-supervised Pretraining](https://mlsanity.com/paper/2308.05734/) (2023)
- [The Song Describer Dataset: a Corpus of Audio Captions for Music-and-Language Evaluation](https://mlsanity.com/paper/2311.10057/) (2023)
- [MusiLingo: Bridging Music and Text with Pre-trained Language Models for Music Captioning and Query Response](https://mlsanity.com/paper/2309.08730/) (2023)
- [MuseChat: A Conversational Music Recommendation System for Videos](https://mlsanity.com/paper/2310.06282/) (2023)
- [A Unified Framework for Multimodal, Multi-Part Human Motion Synthesis](https://mlsanity.com/paper/2311.16471/) (2023)
- [CLaMP 2: Multimodal Music Information Retrieval Across 101 Languages Using Large Language Models](https://mlsanity.com/paper/2410.13267/) (2024)
- [Futga: Towards Fine-grained Music Understanding through Temporally-enhanced Generative Augmentation](https://mlsanity.com/paper/2407.20445/) (2024)
- [I can listen but cannot read: An evaluation of two-tower multimodal systems for instrument recognition](https://mlsanity.com/paper/2407.18058/) (2024)
- [AGAV-Rater: Adapting Large Multimodal Model for AI-Generated Audio-Visual Quality Assessment](https://mlsanity.com/paper/2501.18314/) (2025)
- [WikiMuTe: A web-sourced dataset of semantic descriptions for music audio](https://mlsanity.com/paper/2312.09207/) (2023)
- [Augment, Drop & Swap: Improving Diversity in LLM Captions for Efficient Music-Text Representation Learning](https://mlsanity.com/paper/2409.11498/) (2024)
- [PIAST: A Multimodal Piano Dataset with Audio, Symbolic and Text](https://mlsanity.com/paper/2411.02551/) (2024)
- [MCA: Modality Composition Awareness for Robust Composed Multimodal Retrieval](https://mlsanity.com/paper/2510.15543/) (2025)
- [MDSC: Towards Evaluating the Style Consistency Between Music and Dance](https://mlsanity.com/paper/2309.01340/) (2023)
- [Towards Effective Negation Modeling in Joint Audio-Text Models for Music](https://mlsanity.com/paper/2601.13931/) (2026)
- [MMVA: Multimodal Matching Based on Valence and Arousal across Images, Music, and Musical Captions](https://mlsanity.com/paper/2501.01094/) (2025)
- [MUST-RAG: MUSical Text Question Answering with Retrieval Augmented Generation](https://mlsanity.com/paper/2507.23334/) (2025)
- [Beyond Musical Descriptors: Extracting Preference-Bearing Intent in Music Queries](https://mlsanity.com/paper/2602.12301/) (2026)

## Abstract (arXiv)

This paper introduces effective design choices for text-to-music retrieval systems. An ideal text-based retrieval system would support various input queries such as pre-defined tags, unseen tags, and sentence-level descriptions. In reality, most previous works mainly focused on a single query type (tag or sentence) which may not generalize to another input type. Hence, we review recent text-based music retrieval systems using our proposed benchmark in two main aspects: input text representation and training objectives. Our findings enable a universal text-to-music retrieval system that achieves comparable retrieval performances in both tag- and sentence-level inputs. Furthermore, the proposed multimodal representation generalizes to 9 different downstream music classification tasks. We present the code and demo online.