# Learning Music Audio Representations With Limited Data

arXiv: https://arxiv.org/abs/2505.06042 · HTML: https://mlsanity.com/paper/2505.06042/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 20 of its 31 references that are in the map.
- First posted on arXiv 9 May 2025.

## Builds on (most cited first, 20 of 20)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [An Empirical Study of Training Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.02057/) (2021)
- [AST: Audio Spectrogram Transformer](https://mlsanity.com/paper/2104.01778/) (2021)
- [Deep Convolutional Neural Networks and Data Augmentation for Environmental Sound Classification](https://mlsanity.com/paper/1608.04363/) (2016)
- [Contrastive Learning of Musical Representations](https://mlsanity.com/paper/2103.09410/) (2021)
- [musicnn: Pre-trained convolutional neural networks for music audio tagging](https://mlsanity.com/paper/1909.06654/) (2019)
- [Evaluation of CNN-based Automatic Music Tagging Models](https://mlsanity.com/paper/2006.00751/) (2020)
- [A Survey on Reservoir Computing and its Interdisciplinary Applications Beyond Traditional Machine Learning](https://mlsanity.com/paper/2307.15092/) (2023)
- [Reservoir Transformers](https://mlsanity.com/paper/2012.15045/) (2020)
- [Leveraging Hierarchical Structures for Few-Shot Musical Instrument Recognition](https://mlsanity.com/paper/2107.07029/) (2021)
- [Sample-level CNN Architectures for Music Auto-tagging Using Raw Waveforms](https://mlsanity.com/paper/1710.10451/) (2017)
- [Training neural audio classifiers with few data](https://mlsanity.com/paper/1810.10274/) (2018)
- [Few-Shot Musical Source Separation](https://mlsanity.com/paper/2205.01273/) (2022)
- [Randomly weighted CNNs for (music) audio classification](https://mlsanity.com/paper/1805.00237/) (2018)
- [mir_ref: A Representation Evaluation Framework for Music Information Retrieval Tasks](https://mlsanity.com/paper/2312.05994/) (2023)
- [From West to East: Who can understand the music of the others better?](https://mlsanity.com/paper/2307.09795/) (2023)
- [Modality-Agnostic Self-Supervised Learning with Meta-Learned Masked Auto-Encoder](https://mlsanity.com/paper/2310.16318/) (2023)
- [OrchideaSOL: a dataset of extended instrumental techniques for computer-aided orchestration](https://mlsanity.com/paper/2007.00763/) (2020)

## Abstract (arXiv)

Large deep-learning models for music, including those focused on learning general-purpose music audio representations, are often assumed to require substantial training data to achieve high performance. If true, this would pose challenges in scenarios where audio data or annotations are scarce, such as for underrepresented music traditions, non-popular genres, and personalized music creation and listening. Understanding how these models behave in limited-data scenarios could be crucial for developing techniques to tackle them. In this work, we investigate the behavior of several music audio representation models under limited-data learning regimes. We consider music models with various architectures, training paradigms, and input durations, and train them on data collections ranging from 5 to 8,000 minutes long. We evaluate the learned representations on various music information retrieval tasks and analyze their robustness to noise. We show that, under certain conditions, representations from limited-data and even random models perform comparably to ones from large-dataset models, though handcrafted features outperform all learned representations in some tasks.