# A Functional Taxonomy of Music Generation Systems

arXiv: https://arxiv.org/abs/1812.04186 · HTML: https://mlsanity.com/paper/1812.04186/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Dec 2018.

## Built on by (most cited first, 30 of 31)

- [Mustango: Toward Controllable Text-to-Music Generation](https://mlsanity.com/paper/2311.08355/) (2023)
- [Video2Music: Suitable Music Generation from Videos using an Affective Multimodal Transformer model](https://mlsanity.com/paper/2311.00968/) (2023)
- [MidiCaps: A large-scale MIDI dataset with text captions](https://mlsanity.com/paper/2406.02255/) (2024)
- [Text2midi: Generating Symbolic Music from Captions](https://mlsanity.com/paper/2412.16526/) (2024)
- [Generating Lead Sheets with Affect: A Novel Conditional seq2seq Framework](https://mlsanity.com/paper/2104.13056/) (2021)
- [A Domain-Knowledge-Inspired Music Embedding Space and a Novel Attention Mechanism for Symbolic Music Modeling](https://mlsanity.com/paper/2212.00973/) (2022)
- [Hierarchical Recurrent Neural Networks for Conditional Melody Generation with Long-term Structure](https://mlsanity.com/paper/2102.09794/) (2021)
- [Storyfier: Exploring Vocabulary Learning Support with Text Generation Models](https://mlsanity.com/paper/2308.03864/) (2023)
- [Structure-Enhanced Pop Music Generation via Harmony-Aware Learning](https://mlsanity.com/paper/2109.06441/) (2021)
- [A Review of Intelligent Music Generation Systems](https://mlsanity.com/paper/2211.09124/) (2022)
- [Natural Language Processing Methods for Symbolic Music Generation and Information Retrieval: a Survey](https://mlsanity.com/paper/2402.17467/) (2024)
- [Survey on the Evaluation of Generative Models in Music](https://mlsanity.com/paper/2506.05104/) (2025)
- [AI-Based Affective Music Generation Systems: A Review of Methods, and Challenges](https://mlsanity.com/paper/2301.06890/) (2023)
- [Exploring XAI for the Arts: Explaining Latent Space in Generative Music](https://mlsanity.com/paper/2308.05496/) (2023)
- [Conditional Drums Generation using Compound Word Representations](https://mlsanity.com/paper/2202.04464/) (2022)
- [Exploring Variational Auto-Encoder Architectures, Configurations, and Datasets for Generative Music Explainable AI](https://mlsanity.com/paper/2311.08336/) (2023)
- [A Comprehensive Survey on Generative AI for Video-to-Music Generation](https://mlsanity.com/paper/2502.12489/) (2025)
- [jazznet: A Dataset of Fundamental Piano Patterns for Music Audio Machine Learning Research](https://mlsanity.com/paper/2302.08632/) (2023)
- [An Comparative Analysis of Different Pitch and Metrical Grid Encoding Methods in the Task of Sequential Music Generation](https://mlsanity.com/paper/2301.13383/) (2023)
- [BandCondiNet: Parallel Transformers-based Conditional Popular Music Generation with Multi-View Features](https://mlsanity.com/paper/2407.10462/) (2024)
- [Text Conditioned Symbolic Drumbeat Generation using Latent Diffusion Models](https://mlsanity.com/paper/2408.02711/) (2024)
- [Applications and Advances of Artificial Intelligence in Music Generation:A Review](https://mlsanity.com/paper/2409.03715/) (2024)
- [Musical Speech: A Transformer-based Composition Tool](https://mlsanity.com/paper/2108.01043/) (2021)
- [AffectMachine-Classical: A novel system for generating affective classical music](https://mlsanity.com/paper/2304.04915/) (2023)
- [Prevailing Research Areas for Music AI in the Era of Foundation Models](https://mlsanity.com/paper/2409.09378/) (2024)
- [aiSTROM -- A roadmap for developing a successful AI strategy](https://mlsanity.com/paper/2107.06071/) (2021)
- [ComposeOn Academy: Transforming Melodic Ideas into Complete Compositions Integrating Music Learning](https://mlsanity.com/paper/2502.15255/) (2025)
- [Apollo: An Interactive Environment for Generating Symbolic Musical Phrases using Corpus-based Style Imitation](https://mlsanity.com/paper/2504.14055/) (2025)
- [Aligning Generative Music AI with Human Preferences: Methods and Challenges](https://mlsanity.com/paper/2511.15038/) (2025)
- [Perception of AI-Generated Music -- The Role of Composer Identity, Personality Traits, Music Preferences, and Perceived Humanness](https://mlsanity.com/paper/2512.02785/) (2025)

## Abstract (arXiv)

Digital advances have transformed the face of automatic music generation since its beginnings at the dawn of computing. Despite the many breakthroughs, issues such as the musical tasks targeted by different machines and the degree to which they succeed remain open questions. We present a functional taxonomy for music generation systems with reference to existing systems. The taxonomy organizes systems according to the purposes for which they were designed. It also reveals the inter-relatedness amongst the systems. This design-centered approach contrasts with predominant methods-based surveys and facilitates the identification of grand challenges to set the stage for new breakthroughs.