# Self-Supervised Contrastive Learning for Robust Audio-Sheet Music Retrieval Systems

arXiv: https://arxiv.org/abs/2309.12134 · HTML: https://mlsanity.com/paper/2309.12134/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 43 references that are in the map.
- First posted on arXiv 21 Sep 2023.

## Builds on (most cited first, 20 of 20)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](https://mlsanity.com/paper/1511.07289/) (2015)
- [Discriminative Unsupervised Feature Learning with Exemplar Convolutional Neural Networks](https://mlsanity.com/paper/1406.6909/) (2014)
- [Enabling Factorized Piano Music Modeling and Generation with the MAESTRO Dataset](https://mlsanity.com/paper/1810.12247/) (2018)
- [Unifying Visual-Semantic Embeddings with Multimodal Neural Language Models](https://mlsanity.com/paper/1411.2539/) (2014)
- [Deep Convolutional Neural Networks and Data Augmentation for Environmental Sound Classification](https://mlsanity.com/paper/1608.04363/) (2016)
- [Understanding Optical Music Recognition](https://mlsanity.com/paper/1908.03608/) (2019)
- [Optical Music Recognition with Convolutional Sequence-to-Sequence Models](https://mlsanity.com/paper/1707.04877/) (2017)
- [End-to-End Cross-Modality Retrieval with CCA Projections and Pairwise Ranking Loss](https://mlsanity.com/paper/1705.06979/) (2017)
- [Learning Audio - Sheet Music Correspondences for Score Identification and Offline Alignment](https://mlsanity.com/paper/1707.09887/) (2017)
- [Cross-Modal Music Retrieval and Applications: An Overview of Key Methodologies](https://mlsanity.com/paper/1902.04397/) (2019)
- [Learning Soft-Attention Models for Tempo-invariant Audio-Sheet Music Retrieval](https://mlsanity.com/paper/1906.10996/) (2019)
- [MIDI Passage Retrieval Using Cell Phone Pictures of Sheet Music](https://mlsanity.com/paper/2004.10347/) (2020)
- [A Convolutional-Attentional Neural Framework for Structure-Aware Performance-Score Synchronization](https://mlsanity.com/paper/2204.08822/) (2022)
- [Towards Linking the Lakh and IMSLP Datasets](https://mlsanity.com/paper/2004.10391/) (2020)

## Built on by (most cited first, 1 of 1)

- [Local Multimodal Music Alignment from Global Supervision](https://mlsanity.com/paper/2607.10023/) (2026)

## Abstract (arXiv)

Linking sheet music images to audio recordings remains a key problem for the development of efficient cross-modal music retrieval systems. One of the fundamental approaches toward this task is to learn a cross-modal embedding space via deep neural networks that is able to connect short snippets of audio and sheet music. However, the scarcity of annotated data from real musical content affects the capability of such methods to generalize to real retrieval scenarios. In this work, we investigate whether we can mitigate this limitation with self-supervised contrastive learning, by exposing a network to a large amount of real music data as a pre-training step, by contrasting randomly augmented views of snippets of both modalities, namely audio and sheet images. Through a number of experiments on synthetic and real piano data, we show that pre-trained models are able to retrieve snippets with better precision in all scenarios and pre-training configurations. Encouraged by these results, we employ the snippet embeddings in the higher-level task of cross-modal piece identification and conduct more experiments on several retrieval configurations. In this task, we observe that the retrieval quality improves from 30% up to 100% when real music data is present. We then conclude by arguing for the potential of self-supervised contrastive learning for alleviating the annotated data scarcity in multi-modal music retrieval models.