# Parameter-Efficient Transfer Learning for Music Foundation Models

arXiv: https://arxiv.org/abs/2411.19371 · HTML: https://mlsanity.com/paper/2411.19371/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Nov 2024.

## Built on by (most cited first, 3 of 3)

- [Transcribing Rhythmic Patterns of the Guitar Track in Polyphonic Music](https://mlsanity.com/paper/2510.05756/) (2025)
- [MindMelody: A Closed-Loop EEG-Driven System for Personalized Music Intervention](https://mlsanity.com/paper/2605.01235/) (2026)
- [CoJEPA: Combining Contrastive Learning and JEPA for Global-Local Music Representations](https://mlsanity.com/paper/2608.30974/) (2026)

## Abstract (arXiv)

More music foundation models are recently being released, promising a general, mostly task independent encoding of musical information. Common ways of adapting music foundation models to downstream tasks are probing and fine-tuning. These common transfer learning approaches, however, face challenges. Probing might lead to suboptimal performance because the pre-trained weights are frozen, while fine-tuning is computationally expensive and is prone to overfitting. Our work investigates the use of parameter-efficient transfer learning (PETL) for music foundation models which integrates the advantage of probing and fine-tuning. We introduce three types of PETL methods: adapter-based methods, prompt-based methods, and reparameterization-based methods. These methods train only a small number of parameters, and therefore do not require significant computational resources. Results show that PETL methods outperform both probing and fine-tuning on music auto-tagging. On key detection and tempo estimation, they achieve similar results as fine-tuning with significantly less training cost. However, the usefulness of the current generation of foundation model on key and tempo tasks is questioned by the similar results achieved by training a small model from scratch. Code available at https://github.com/suncerock/peft-music/