# Symbolic music generation conditioned on continuous-valued emotions

arXiv: https://arxiv.org/abs/2203.16165 · HTML: https://mlsanity.com/paper/2203.16165/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 71 references that are in the map.
- First posted on arXiv 30 Mar 2022.

## Builds on (most cited first, 30 of 37)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [Show, Attend and Tell: Neural Image Caption Generation with Visual Attention](https://mlsanity.com/paper/1502.03044/) (2015)
- [On the Properties of Neural Machine Translation: Encoder-Decoder Approaches](https://mlsanity.com/paper/1409.1259/) (2014)
- [Plug and Play Language Models: A Simple Approach to Controlled Text Generation](https://mlsanity.com/paper/1912.02164/) (2019)
- [CTRL: A Conditional Transformer Language Model for Controllable Generation](https://mlsanity.com/paper/1909.05858/) (2019)
- [Jukebox: A Generative Model for Music](https://mlsanity.com/paper/2005.00341/) (2020)
- [Enabling Factorized Piano Music Modeling and Generation with the MAESTRO Dataset](https://mlsanity.com/paper/1810.12247/) (2018)
- [Music Transformer](https://mlsanity.com/paper/1809.04281/) (2018)
- [Reformulating Unsupervised Style Transfer as Paraphrase Generation](https://mlsanity.com/paper/2010.05700/) (2020)
- [MuseGAN: Multi-track Sequential Generative Adversarial Networks for Symbolic Music Generation and Accompaniment](https://mlsanity.com/paper/1709.06298/) (2017)
- [A Hierarchical Latent Vector Model for Learning Long-Term Structure in Music](https://mlsanity.com/paper/1803.05428/) (2018)
- [Show and Tell: Lessons learned from the 2015 MSCOCO Image Captioning Challenge](https://mlsanity.com/paper/1609.06647/) (2016)
- [Towards Controllable Biases in Language Generation](https://mlsanity.com/paper/2005.00268/) (2020)
- [SampleRNN: An Unconditional End-to-End Neural Audio Generation Model](https://mlsanity.com/paper/1612.07837/) (2016)
- [Retrieve and Refine: Improved Sequence Generation Models For Dialogue](https://mlsanity.com/paper/1808.04776/) (2018)
- [MidiNet: A Convolutional Generative Adversarial Network for Symbolic-domain Music Generation](https://mlsanity.com/paper/1703.10847/) (2017)
- [This Time with Feeling: Learning Expressive Musical Performance](https://mlsanity.com/paper/1808.03715/) (2018)
- [LakhNES: Improving multi-instrumental music generation with cross-domain pre-training](https://mlsanity.com/paper/1907.04868/) (2019)
- [EMOPIA: A Multi-Modal Pop Piano Dataset For Emotion Recognition and Emotion-based Music Generation](https://mlsanity.com/paper/2108.01374/) (2021)
- [Music FaderNets: Controllable Music Generation Based On High-Level Features via Low-Level Feature Modelling](https://mlsanity.com/paper/2007.15474/) (2020)
- [Counterpoint by Convolution](https://mlsanity.com/paper/1903.07227/) (2019)
- [Learning Interpretable Representation for Controllable Polyphonic Music Generation](https://mlsanity.com/paper/2008.07122/) (2020)
- [Controlling Style in Generated Dialogue](https://mlsanity.com/paper/2009.10855/) (2020)

## Built on by (most cited first, 13 of 13)

- [JEN-1: Text-Guided Universal Music Generation with Omnidirectional Diffusion Models](https://mlsanity.com/paper/2308.04729/) (2023)
- [Natural Language Processing Methods for Symbolic Music Generation and Information Retrieval: a Survey](https://mlsanity.com/paper/2402.17467/) (2024)
- [AI-Based Affective Music Generation Systems: A Review of Methods, and Challenges](https://mlsanity.com/paper/2301.06890/) (2023)
- [REMAST: Real-time Emotion-based Music Arrangement with Soft Transition](https://mlsanity.com/paper/2305.08029/) (2023)
- [EmoGen: Eliminating Subjective Bias in Emotional Music Generation](https://mlsanity.com/paper/2307.01229/) (2023)
- [Emotion4MIDI: a Lyrics-based Emotion-Labeled Symbolic Music Dataset](https://mlsanity.com/paper/2307.14783/) (2023)
- [Video Soundtrack Generation by Aligning Emotions and Temporal Boundaries](https://mlsanity.com/paper/2502.10154/) (2025)
- [EMelodyGen: Emotion-Conditioned Melody Generation in ABC Notation with the Musical Feature Template](https://mlsanity.com/paper/2309.13259/) (2023)
- [On the de-duplication of the Lakh MIDI dataset](https://mlsanity.com/paper/2509.16662/) (2025)
- [LooPy: A Research-Friendly Mix Framework for Music Information Retrieval on Electronic Dance Music](https://mlsanity.com/paper/2305.01051/) (2023)
- [Addressing Emotion Bias in Music Emotion Recognition and Generation with Frechet Audio Distance](https://mlsanity.com/paper/2409.15545/) (2024)
- [AImoclips: A Benchmark for Evaluating Emotion Conveyance in Text-to-Music Generation](https://mlsanity.com/paper/2509.00813/) (2025)
- [Self-Representation Alignment in Diffusion Models for Video Background Music Generation](https://mlsanity.com/paper/2610.15575/) (2026)

## Abstract (arXiv)

In this paper we present a new approach for the generation of multi-instrument symbolic music driven by musical emotion. The principal novelty of our approach centres on conditioning a state-of-the-art transformer based on continuous-valued valence and arousal labels. In addition, we provide a new large-scale dataset of symbolic music paired with emotion labels in terms of valence and arousal. We evaluate our approach in a quantitative manner in two ways, first by measuring its note prediction accuracy, and second via a regression task in the valence-arousal plane. Our results demonstrate that our proposed approaches outperform conditioning using control tokens which is representative of the current state of the art.