# AI-Based Affective Music Generation Systems: A Review of Methods, and Challenges

arXiv: https://arxiv.org/abs/2301.06890 · HTML: https://mlsanity.com/paper/2301.06890/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 119 references that are in the map.
- First posted on arXiv 10 Jan 2023.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling](https://mlsanity.com/paper/1412.3555/) (2014)
- [Compound Word Transformer: Learning to Compose Full-Song Music over Dynamic Directed Hypergraphs](https://mlsanity.com/paper/2101.02402/) (2021)
- [A Comprehensive Survey on Deep Music Generation: Multi-level Representations, Algorithms, Evaluations, and Future Directions](https://mlsanity.com/paper/2011.06801/) (2020)
- [EMOPIA: A Multi-Modal Pop Piano Dataset For Emotion Recognition and Emotion-based Music Generation](https://mlsanity.com/paper/2108.01374/) (2021)
- [A Functional Taxonomy of Music Generation Systems](https://mlsanity.com/paper/1812.04186/) (2018)
- [Learning to Generate Music With Sentiment](https://mlsanity.com/paper/2103.06125/) (2021)
- [Music Generation by Deep Learning - Challenges and Directions](https://mlsanity.com/paper/1712.04371/) (2017)
- [Computer-Generated Music for Tabletop Role-Playing Games](https://mlsanity.com/paper/2008.07009/) (2020)
- [MorpheuS: generating structured music with constrained patterns and tension](https://mlsanity.com/paper/1812.04832/) (2018)
- [Generating Lead Sheets with Affect: A Novel Conditional seq2seq Framework](https://mlsanity.com/paper/2104.13056/) (2021)
- [Symbolic music generation conditioned on continuous-valued emotions](https://mlsanity.com/paper/2203.16165/) (2022)
- [Controlling Perceived Emotion in Symbolic Music Generation with Monte Carlo Tree Search](https://mlsanity.com/paper/2208.05162/) (2022)
- [Generating music with sentiment using Transformer-GANs](https://mlsanity.com/paper/2212.11134/) (2022)
- [Adaptive Music Composition for Games](https://mlsanity.com/paper/1907.01154/) (2019)
- [Predicting emotion from music videos: exploring the relative contribution of visual and auditory information to affective responses](https://mlsanity.com/paper/2202.10453/) (2022)
- [Artificial Musical Intelligence: A Survey](https://mlsanity.com/paper/2006.10553/) (2020)
- [EmotionBox: a music-element-driven emotional music generation system using Recurrent Neural Network](https://mlsanity.com/paper/2112.08561/) (2021)

## Built on by (most cited first, 9 of 9)

- [Are We There Yet? A Brief Survey of Music Emotion Prediction Datasets, Models and Outstanding Challenges](https://mlsanity.com/paper/2406.08809/) (2024)
- [A Comprehensive Survey on Generative AI for Video-to-Music Generation](https://mlsanity.com/paper/2502.12489/) (2025)
- [A Survey on Music Generation from Single-Modal, Cross-Modal, and Multi-Modal Perspectives](https://mlsanity.com/paper/2504.00837/) (2025)
- [SyMuPe: Affective and Controllable Symbolic Music Performance](https://mlsanity.com/paper/2511.03425/) (2025)
- [AffectMachine-Classical: A novel system for generating affective classical music](https://mlsanity.com/paper/2304.04915/) (2023)
- [Once Upon an AI: Six Scaffolds for Child-AI Interaction Design, Inspired by Disney](https://mlsanity.com/paper/2504.08670/) (2025)
- [TinyMusician: On-Device Music Generation with Knowledge Distillation and Mixed Precision Quantization](https://mlsanity.com/paper/2509.00914/) (2025)
- [Story2MIDI: Emotionally Aligned Music Generation from Text](https://mlsanity.com/paper/2512.02192/) (2025)
- [AI-Based Sound Effect Generation: A Narrative Review of Generative Models Across Input Modalities](https://mlsanity.com/paper/2608.03742/) (2026)

## Abstract (arXiv)

Music is a powerful medium for altering the emotional state of the listener. In recent years, with significant advancement in computing capabilities, artificial intelligence-based (AI-based) approaches have become popular for creating affective music generation (AMG) systems that are empowered with the ability to generate affective music. Entertainment, healthcare, and sensor-integrated interactive system design are a few of the areas in which AI-based affective music generation (AI-AMG) systems may have a significant impact. Given the surge of interest in this topic, this article aims to provide a comprehensive review of AI-AMG systems. The main building blocks of an AI-AMG system are discussed, and existing systems are formally categorized based on the core algorithm used for music generation. In addition, this article discusses the main musical features employed to compose affective music, along with the respective AI-based approaches used for tailoring them. Lastly, the main challenges and open questions in this field, as well as their potential solutions, are presented to guide future research. We hope that this review will be useful for readers seeking to understand the state-of-the-art in AI-AMG systems, and gain an overview of the methods used for developing them, thereby helping them explore this field in the future.