# Music Composition with Deep Learning: A Review

arXiv: https://arxiv.org/abs/2108.12290 · HTML: https://mlsanity.com/paper/2108.12290/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 71 references that are in the map.
- First posted on arXiv 27 Aug 2021.

## Builds on (most cited first, 30 of 31)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [A Comprehensive Survey on Transfer Learning](https://mlsanity.com/paper/1911.02685/) (2019)
- [SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient](https://mlsanity.com/paper/1609.05473/) (2016)
- [Music Transformer](https://mlsanity.com/paper/1809.04281/) (2018)
- [Symbolic Music Generation with Diffusion Models](https://mlsanity.com/paper/2103.16091/) (2021)
- [MuseGAN: Multi-track Sequential Generative Adversarial Networks for Symbolic Music Generation and Accompaniment](https://mlsanity.com/paper/1709.06298/) (2017)
- [RSA: Byzantine-Robust Stochastic Aggregation Methods for Distributed Learning from Heterogeneous Datasets](https://mlsanity.com/paper/1811.03761/) (2018)
- [C-RNN-GAN: Continuous recurrent neural networks with adversarial training](https://mlsanity.com/paper/1611.09904/) (2016)
- [A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music](https://mlsanity.com/paper/1803.05428/) (2018)
- [DeepBach: a Steerable Model for Bach Chorales Generation](https://mlsanity.com/paper/1612.01010/) (2016)
- [A Comprehensive Survey on Deep Music Generation: Multi-level Representations, Algorithms, Evaluations, and Future Directions](https://mlsanity.com/paper/2011.06801/) (2020)
- [LakhNES: Improving multi-instrumental music generation with cross-domain pre-training](https://mlsanity.com/paper/1907.04868/) (2019)
- [MMM : Exploring Conditional Multi-Track Music Generation with the Transformer](https://mlsanity.com/paper/2008.06048/) (2020)
- [Counterpoint by Convolution](https://mlsanity.com/paper/1903.07227/) (2019)
- [Learning Interpretable Representation for Controllable Polyphonic Music Generation](https://mlsanity.com/paper/2008.07122/) (2020)
- [MuseMorphose: Full-Song and Fine-Grained Piano Music Style Transfer with One Transformer VAE](https://mlsanity.com/paper/2105.04090/) (2021)
- [MIDI-VAE: Modeling Dynamics and Instrumentation of Music with Applications to Style Transfer](https://mlsanity.com/paper/1809.07600/) (2018)
- [Symbolic Music Genre Transfer with CycleGAN](https://mlsanity.com/paper/1809.07575/) (2018)
- [PIANOTREE VAE: Structured Representation Learning for Polyphonic Music](https://mlsanity.com/paper/2008.07118/) (2020)
- [Convolutional Generative Adversarial Networks with Binary Neurons for Polyphonic Music Generation](https://mlsanity.com/paper/1804.09399/) (2018)
- [Automatic Melody Harmonization with Triad Chords: A Comparative Study](https://mlsanity.com/paper/2001.02360/) (2020)
- [Song From PI: A Musically Plausible Network for Pop Music Generation](https://mlsanity.com/paper/1611.03477/) (2016)
- [Improving Automatic Jazz Melody Generation by Transfer Learning Techniques](https://mlsanity.com/paper/1908.09484/) (2019)
- [Imposing higher-level Structure in Polyphonic Music Generation using Convolutional Restricted Boltzmann Machines and Constraints](https://mlsanity.com/paper/1612.04742/) (2016)
- [Interactive Music Generation with Positional Constraints using Anticipation-RNNs](https://mlsanity.com/paper/1709.06404/) (2017)
- [Improving Polyphonic Music Models with Feature-Rich Encoding](https://mlsanity.com/paper/1911.11775/) (2019)
- [A Unit Selection Methodology for Music Generation Using Deep Neural Networks](https://mlsanity.com/paper/1612.03789/) (2016)
- [Generating Nontrivial Melodies for Music as a Service](https://mlsanity.com/paper/1710.02280/) (2017)
- [Learning to Fuse Music Genres with Generative Adversarial Dual Learning](https://mlsanity.com/paper/1712.01456/) (2017)

## Built on by (most cited first, 12 of 12)

- [SongBloom: Coherent Song Generation via Interleaved Autoregressive Sketching and Diffusion Refinement](https://mlsanity.com/paper/2506.07634/) (2025)
- [A Survey of AI Music Generation Tools and Models](https://mlsanity.com/paper/2308.12982/) (2023)
- [The GigaMIDI Dataset with Features for Expressive Music Performance Detection](https://mlsanity.com/paper/2502.17726/) (2025)
- [Subjective Evaluation of Deep Learning Models for Symbolic Music Composition](https://mlsanity.com/paper/2203.14641/) (2022)
- [EmoGen: Eliminating Subjective Bias in Emotional Music Generation](https://mlsanity.com/paper/2307.01229/) (2023)
- [Vision-to-Music Generation: A Survey](https://mlsanity.com/paper/2503.21254/) (2025)
- [A Survey on Artificial Intelligence for Music Generation: Agents, Domains and Perspectives](https://mlsanity.com/paper/2210.13944/) (2022)
- [Symbolic Music Structure Analysis with Graph Representations and Changepoint Detection Methods](https://mlsanity.com/paper/2303.13881/) (2023)
- [FIGARO: Generating Symbolic Music with Fine-Grained Artistic Control](https://mlsanity.com/paper/2201.10936/) (2022)
- [VCoME: Verbal Video Composition with Multimodal Editing Effects](https://mlsanity.com/paper/2407.04697/) (2024)
- [AI and the creative realm: A short review of current and future applications](https://mlsanity.com/paper/2306.01795/) (2023)
- [The Evolution and Future Perspectives of Artificial Intelligence Generated Content](https://mlsanity.com/paper/2412.01948/) (2024)

## Abstract (arXiv)

Generating a complex work of art such as a musical composition requires exhibiting true creativity that depends on a variety of factors that are related to the hierarchy of musical language. Music generation have been faced with Algorithmic methods and recently, with Deep Learning models that are being used in other fields such as Computer Vision. In this paper we want to put into context the existing relationships between AI-based music composition models and human musical composition and creativity processes. We give an overview of the recent Deep Learning models for music composition and we compare these models to the music composition process from a theoretical point of view. We have tried to answer some of the most relevant open questions for this task by analyzing the ability of current Deep Learning models to generate music with creativity or the similarity between AI and human composition processes, among others.