# CLaMP 3: Universal Music Information Retrieval Across Unaligned Modalities and Unseen Languages

arXiv: https://arxiv.org/abs/2502.10362 · HTML: https://mlsanity.com/paper/2502.10362/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Feb 2025.

## Built on by (most cited first, 21 of 21)

- [Qwen3-Omni Technical Report](https://mlsanity.com/paper/2509.17765/) (2025)
- [Scaling Self-Supervised Representation Learning for Symbolic Piano Performance](https://mlsanity.com/paper/2506.23869/) (2025)
- [CMI-RewardBench: Evaluating Music Reward Models with Compositional Multimodal Instruction](https://mlsanity.com/paper/2603.00610/) (2026)
- [UniMoE-Audio: Unified Speech and Music Generation with Dynamic-Capacity MoE](https://mlsanity.com/paper/2510.13344/) (2025)
- [Composer Vector: Style-steering Symbolic Music Generation in a Latent Space](https://mlsanity.com/paper/2604.03333/) (2026)
- [SymphonyGen: 3D Hierarchical Orchestral Generation with Controllable Harmony Skeleton](https://mlsanity.com/paper/2604.25498/) (2026)
- [On the de-duplication of the Lakh MIDI dataset](https://mlsanity.com/paper/2509.16662/) (2025)
- [Music Recommendation with Large Language Models: Challenges, Opportunities, and Evaluation](https://mlsanity.com/paper/2511.16478/) (2025)
- [MUST-RAG: MUSical Text Question Answering with Retrieval Augmented Generation](https://mlsanity.com/paper/2507.23334/) (2025)
- [EmoHeal: An End-to-End System for Personalized Therapeutic Music Retrieval from Fine-grained Emotions](https://mlsanity.com/paper/2509.15986/) (2025)
- [IncompeBench: A Permissively Licensed, Fine-Grained Benchmark for Music Information Retrieval](https://mlsanity.com/paper/2602.11941/) (2026)
- [Affective Music Recommendation: A Rollout-Based World Model for Offline Preference Optimization](https://mlsanity.com/paper/2605.28810/) (2026)
- [Towards Unified Song Generation and Singing Voice Conversion with Accompaniment Co-Generation](https://mlsanity.com/paper/2606.07015/) (2026)
- [MADB: A Large-Scale Music Aesthetics Dataset with Professional and Multi-Dimensional Annotations](https://mlsanity.com/paper/2607.06929/) (2026)
- [Local Multimodal Music Alignment from Global Supervision](https://mlsanity.com/paper/2607.10023/) (2026)
- [Integrating Contextual Embeddings into Evaluation of Expressive MIDI Piano Performances](https://mlsanity.com/paper/2607.27909/) (2026)
- [Learning Music Style for Piano Arrangement Through Cross-Modal Bootstrapping](https://mlsanity.com/paper/2608.03050/) (2026)
- [MUUNRiver-Bench: Diagnosing Relation-Dependent Music Retrieval with Multimodal Instructions](https://mlsanity.com/paper/2609.16090/) (2026)
- [Don't CLAP: Are Music-Text Models Bag-of-Words?](https://mlsanity.com/paper/2609.30540/) (2026)
- [From Isolated Feature to Orbits: Discovering Music Concepts via Multi-SAE Alignment](https://mlsanity.com/paper/2610.01864/) (2026)
- [Extending Music Annotation Schemas: Zero-Shot Prediction or Few-Shot Adaptation?](https://mlsanity.com/paper/2610.06920/) (2026)

## Abstract (arXiv)

CLaMP 3 is a unified framework developed to address challenges of cross-modal and cross-lingual generalization in music information retrieval. Using contrastive learning, it aligns all major music modalities--including sheet music, performance signals, and audio recordings--with multilingual text in a shared representation space, enabling retrieval across unaligned modalities with text as a bridge. It features a multilingual text encoder adaptable to unseen languages, exhibiting strong cross-lingual generalization. Leveraging retrieval-augmented generation, we curated M4-RAG, a web-scale dataset consisting of 2.31 million music-text pairs. This dataset is enriched with detailed metadata that represents a wide array of global musical traditions. To advance future research, we release WikiMT-X, a benchmark comprising 1,000 triplets of sheet music, audio, and richly varied text descriptions. Experiments show that CLaMP 3 achieves state-of-the-art performance on multiple MIR tasks, significantly surpassing previous strong baselines and demonstrating excellent generalization in multimodal and multilingual music contexts.