# MMT-BERT: Chord-aware Symbolic Music Generation Based on Multitrack Music Transformer and MusicBERT

arXiv: https://arxiv.org/abs/2409.00919 · HTML: https://mlsanity.com/paper/2409.00919/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 37 references that are in the map.
- First posted on arXiv 2 Sep 2024.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [Simple and Controllable Music Generation](https://mlsanity.com/paper/2306.05284/) (2023)
- [The relativistic discriminator: a key element missing from standard GAN](https://mlsanity.com/paper/1807.00734/) (2018)
- [MuseGAN: Multi-track Sequential Generative Adversarial Networks for Symbolic Music Generation and Accompaniment](https://mlsanity.com/paper/1709.06298/) (2017)
- [C-RNN-GAN: Continuous recurrent neural networks with adversarial training](https://mlsanity.com/paper/1611.09904/) (2016)
- [Compound Word Transformer: Learning to Compose Full-Song Music over Dynamic Directed Hypergraphs](https://mlsanity.com/paper/2101.02402/) (2021)
- [MusicBERT: Symbolic Music Understanding with Large-Scale Pre-Training](https://mlsanity.com/paper/2106.05630/) (2021)
- [Jointly Measuring Diversity and Quality in Text Generation Models](https://mlsanity.com/paper/1904.03971/) (2019)
- [A Comprehensive Survey on Deep Music Generation: Multi-level Representations, Algorithms, Evaluations, and Future Directions](https://mlsanity.com/paper/2011.06801/) (2020)
- [The Jazz Transformer on the Front Line: Exploring the Shortcomings of AI-composed Music through Quantitative Measures](https://mlsanity.com/paper/2008.01307/) (2020)
- [Deep Learning Techniques for Music Generation -- A Survey](https://mlsanity.com/paper/1709.01620/) (2017)
- [MMM : Exploring Conditional Multi-Track Music Generation with the Transformer](https://mlsanity.com/paper/2008.06048/) (2020)
- [Museformer: Transformer with Fine- and Coarse-Grained Attention for Music Generation](https://mlsanity.com/paper/2210.10349/) (2022)
- [MusPy: A Toolkit for Symbolic Music Generation](https://mlsanity.com/paper/2008.01951/) (2020)
- [Multitrack Music Transformer](https://mlsanity.com/paper/2207.06983/) (2022)
- [Convolutional Generative Adversarial Networks with Binary Neurons for Polyphonic Music Generation](https://mlsanity.com/paper/1804.09399/) (2018)
- [A Bi-directional Transformer for Musical Chord Recognition](https://mlsanity.com/paper/1907.02698/) (2019)
- [A database linking piano and orchestral MIDI scores with application to automatic projective orchestration](https://mlsanity.com/paper/1810.08611/) (2018)
- [Generating music with sentiment using Transformer-GANs](https://mlsanity.com/paper/2212.11134/) (2022)

## Built on by (most cited first, 1 of 1)

- [SG-Tailor: Inter-Object Commonsense Relationship Reasoning for Scene Graph Manipulation](https://mlsanity.com/paper/2503.18988/) (2025)

## Abstract (arXiv)

We propose a novel symbolic music representation and Generative Adversarial Network (GAN) framework specially designed for symbolic multitrack music generation. The main theme of symbolic music generation primarily encompasses the preprocessing of music data and the implementation of a deep learning framework. Current techniques dedicated to symbolic music generation generally encounter two significant challenges: training data's lack of information about chords and scales and the requirement of specially designed model architecture adapted to the unique format of symbolic music representation. In this paper, we solve the above problems by introducing new symbolic music representation with MusicLang chord analysis model. We propose our MMT-BERT architecture adapting to the representation. To build a robust multitrack music generator, we fine-tune a pre-trained MusicBERT model to serve as the discriminator, and incorporate relativistic standard loss. This approach, supported by the in-depth understanding of symbolic music encoded within MusicBERT, fortifies the consonance and humanity of music generated by our method. Experimental results demonstrate the effectiveness of our approach which strictly follows the state-of-the-art methods.