# Systematic Analysis of Music Representations from BERT

arXiv: https://arxiv.org/abs/2306.04628 · HTML: https://mlsanity.com/paper/2306.04628/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Jun 2023.

## Built on by (most cited first, 2 of 2)

- [Natural Language Processing Methods for Symbolic Music Generation and Information Retrieval: a Survey](https://mlsanity.com/paper/2402.17467/) (2024)
- [On the de-duplication of the Lakh MIDI dataset](https://mlsanity.com/paper/2509.16662/) (2025)

## Abstract (arXiv)

There have been numerous attempts to represent raw data as numerical vectors that effectively capture semantic and contextual information. However, in the field of symbolic music, previous works have attempted to validate their music embeddings by observing the performance improvement of various fine-tuning tasks. In this work, we directly analyze embeddings from BERT and BERT with contrastive learning trained on bar-level MIDI, inspecting their musical information that can be obtained from MIDI events. We observe that the embeddings exhibit distinct characteristics of information depending on the contrastive objectives and the choice of layers. Our code is available at https://github.com/sjhan91/MusicBERT.