# BumbleBee: A Transformer for Music

arXiv: https://arxiv.org/abs/2107.03443 · HTML: https://mlsanity.com/paper/2107.03443/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 10 references that are in the map.
- First posted on arXiv 7 Jul 2021.

## Builds on (most cited first, 9 of 9)

- [Longformer: The Long-Document Transformer](https://mlsanity.com/paper/2004.05150/) (2020)
- [Jukebox: A Generative Model for Music](https://mlsanity.com/paper/2005.00341/) (2020)
- [Music Transformer](https://mlsanity.com/paper/1809.04281/) (2018)
- [MuseGAN: Multi-track Sequential Generative Adversarial Networks for Symbolic Music Generation and Accompaniment](https://mlsanity.com/paper/1709.06298/) (2017)
- [A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music](https://mlsanity.com/paper/1803.05428/) (2018)
- [DeepBach: a Steerable Model for Bach Chorales Generation](https://mlsanity.com/paper/1612.01010/) (2016)
- [MidiNet: A Convolutional Generative Adversarial Network for Symbolic-domain Music Generation](https://mlsanity.com/paper/1703.10847/) (2017)
- [Counterpoint by Convolution](https://mlsanity.com/paper/1903.07227/) (2019)
- [A Hierarchical Recurrent Neural Network for Symbolic Melody Generation](https://mlsanity.com/paper/1712.05274/) (2017)

## Built on by (most cited first, 1 of 1)

- [The Ethical Implications of Generative Audio Models: A Systematic Literature Review](https://mlsanity.com/paper/2307.05527/) (2023)

## Abstract (arXiv)

We will introduce BumbleBee, a transformer model that will generate MIDI music data . We will tackle the issue of transformers applied to long sequences by implementing a longformer generative model that uses dilating sliding windows to compute the attention layers. We will compare our results to that of the music transformer and Long-Short term memory (LSTM) to benchmark our results. This analysis will be performed using piano MIDI files, in particular , the JSB Chorales dataset that has already been used for other research works (Huang et al., 2018)