# Generating Lead Sheets with Affect: A Novel Conditional seq2seq Framework

arXiv: https://arxiv.org/abs/2104.13056 · HTML: https://mlsanity.com/paper/2104.13056/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 45 references that are in the map.
- First posted on arXiv 27 Apr 2021.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Effective Approaches to Attention-based Neural Machine Translation](https://mlsanity.com/paper/1508.04025/) (2015)
- [Generalized Cross Entropy Loss for Training Deep Neural Networks with Noisy Labels](https://mlsanity.com/paper/1805.07836/) (2018)
- [Music Transformer](https://mlsanity.com/paper/1809.04281/) (2018)
- [MuseGAN: Multi-track Sequential Generative Adversarial Networks for Symbolic Music Generation and Accompaniment](https://mlsanity.com/paper/1709.06298/) (2017)
- [DeepBach: a Steerable Model for Bach Chorales Generation](https://mlsanity.com/paper/1612.01010/) (2016)
- [This Time with Feeling: Learning Expressive Musical Performance](https://mlsanity.com/paper/1808.03715/) (2018)
- [MIDI-VAE: Modeling Dynamics and Instrumentation of Music with Applications to Style Transfer](https://mlsanity.com/paper/1809.07600/) (2018)
- [A Functional Taxonomy of Music Generation Systems](https://mlsanity.com/paper/1812.04186/) (2018)
- [Learning to Generate Music With Sentiment](https://mlsanity.com/paper/2103.06125/) (2021)
- [MorpheuS: generating structured music with constrained patterns and tension](https://mlsanity.com/paper/1812.04832/) (2018)
- [Hierarchical Recurrent Neural Networks for Conditional Melody Generation with Long-term Structure](https://mlsanity.com/paper/2102.09794/) (2021)
- [Regression-based music emotion prediction using triplet neural networks](https://mlsanity.com/paper/2001.09988/) (2020)
- [Lead Sheet Generation and Arrangement by Conditional Generative Adversarial Network](https://mlsanity.com/paper/1807.11161/) (2018)
- [Rhythm, Chord and Melody Generation for Lead Sheets using Recurrent Neural Networks](https://mlsanity.com/paper/2002.10266/) (2020)

## Built on by (most cited first, 17 of 17)

- [Video2Music: Suitable Music Generation from Videos using an Affective Multimodal Transformer model](https://mlsanity.com/paper/2311.00968/) (2023)
- [Text2midi: Generating Symbolic Music from Captions](https://mlsanity.com/paper/2412.16526/) (2024)
- [Generating music with sentiment using Transformer-GANs](https://mlsanity.com/paper/2212.11134/) (2022)
- [Natural Language Processing Methods for Symbolic Music Generation and Information Retrieval: a Survey](https://mlsanity.com/paper/2402.17467/) (2024)
- [AI-Based Affective Music Generation Systems: A Review of Methods, and Challenges](https://mlsanity.com/paper/2301.06890/) (2023)
- [Conditional Drums Generation using Compound Word Representations](https://mlsanity.com/paper/2202.04464/) (2022)
- [Are We There Yet? A Brief Survey of Music Emotion Prediction Datasets, Models and Outstanding Challenges](https://mlsanity.com/paper/2406.08809/) (2024)
- [Towards Unified Music Emotion Recognition across Dimensional and Categorical Models](https://mlsanity.com/paper/2502.03979/) (2025)
- [Predicting emotion from music videos: exploring the relative contribution of visual and auditory information to affective responses](https://mlsanity.com/paper/2202.10453/) (2022)
- [Emotion-Driven Melody Harmonization via Melodic Variation and Functional Representation](https://mlsanity.com/paper/2407.20176/) (2024)
- [Forecasting Bitcoin volatility spikes from whale transactions and CryptoQuant data using Synthesizer Transformer models](https://mlsanity.com/paper/2211.08281/) (2022)
- [Unifying Symbolic Music Arrangement: Track-Aware Reconstruction and Structured Tokenization](https://mlsanity.com/paper/2408.15176/) (2024)
- [Emotion-Conditioned Melody Harmonization with Hierarchical Variational Autoencoder](https://mlsanity.com/paper/2306.03718/) (2023)
- [MusER: Musical Element-Based Regularization for Generating Symbolic Music with Emotion](https://mlsanity.com/paper/2312.10307/) (2023)
- [Prevailing Research Areas for Music AI in the Era of Foundation Models](https://mlsanity.com/paper/2409.09378/) (2024)
- [Unsupervised Lead Sheet Generation via Semantic Compression](https://mlsanity.com/paper/2310.10772/) (2023)
- [Explicit Tonal Tension Conditioning via Dual-Level Beam Search for Symbolic Music Generation](https://mlsanity.com/paper/2511.19342/) (2025)

## Abstract (arXiv)

The field of automatic music composition has seen great progress in the last few years, much of which can be attributed to advances in deep neural networks. There are numerous studies that present different strategies for generating sheet music from scratch. The inclusion of high-level musical characteristics (e.g., perceived emotional qualities), however, as conditions for controlling the generation output remains a challenge. In this paper, we present a novel approach for calculating the valence (the positivity or negativity of the perceived emotion) of a chord progression within a lead sheet, using pre-defined mood tags proposed by music experts. Based on this approach, we propose a novel strategy for conditional lead sheet generation that allows us to steer the music generation in terms of valence, phrasing, and time signature. Our approach is similar to a Neural Machine Translation (NMT) problem, as we include high-level conditions in the encoder part of the sequence-to-sequence architectures used (i.e., long-short term memory networks, and a Transformer network). We conducted experiments to thoroughly analyze these two architectures. The results show that the proposed strategy is able to generate lead sheets in a controllable manner, resulting in distributions of musical attributes similar to those of the training dataset. We also verified through a subjective listening test that our approach is effective in controlling the valence of a generated chord progression.