# Emotion-Driven Melody Harmonization via Melodic Variation and Functional Representation

arXiv: https://arxiv.org/abs/2407.20176 · HTML: https://mlsanity.com/paper/2407.20176/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 31 references that are in the map.
- First posted on arXiv 29 Jul 2024.

## Builds on (most cited first, 17 of 17)

- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Rethinking Attention with Performers](https://mlsanity.com/paper/2009.14794/) (2020)
- [CTRL: A Conditional Transformer Language Model for Controllable Generation](https://mlsanity.com/paper/1909.05858/) (2019)
- [Music Transformer](https://mlsanity.com/paper/1809.04281/) (2018)
- [Compound Word Transformer: Learning to Compose Full-Song Music over Dynamic Directed Hypergraphs](https://mlsanity.com/paper/2101.02402/) (2021)
- [EMOPIA: A Multi-Modal Pop Piano Dataset For Emotion Recognition and Emotion-based Music Generation](https://mlsanity.com/paper/2108.01374/) (2021)
- [MuseCoco: Generating Symbolic Music from Text](https://mlsanity.com/paper/2306.00110/) (2023)
- [Learning to Generate Music With Sentiment](https://mlsanity.com/paper/2103.06125/) (2021)
- [Melody transcription via generative pre-training](https://mlsanity.com/paper/2212.01884/) (2022)
- [Automatic Melody Harmonization with Triad Chords: A Comparative Study](https://mlsanity.com/paper/2001.02360/) (2020)
- [Compose & Embellish: Well-Structured Piano Performance Generation via A Two-Stage Approach](https://mlsanity.com/paper/2209.08212/) (2022)
- [Generating Lead Sheets with Affect: A Novel Conditional seq2seq Framework](https://mlsanity.com/paper/2104.13056/) (2021)
- [SurpriseNet: Melody Harmonization Conditioning on User-controlled Surprise Contours](https://mlsanity.com/paper/2108.00378/) (2021)
- [Melody Harmonization Using Orderless NADE, Chord Balancing, and Blocked Gibbs Sampling](https://mlsanity.com/paper/2010.13468/) (2020)
- [EmoGen: Eliminating Subjective Bias in Emotional Music Generation](https://mlsanity.com/paper/2307.01229/) (2023)
- [Domain Adversarial Training on Conditional Variational Auto-Encoder for Controllable Music Generation](https://mlsanity.com/paper/2209.07144/) (2022)
- [Emotion-Conditioned Melody Harmonization with Hierarchical Variational Autoencoder](https://mlsanity.com/paper/2306.03718/) (2023)

## Built on by (most cited first, 4 of 4)

- [Emotion-driven Piano Music Generation via Two-stage Disentanglement and Functional Representation](https://mlsanity.com/paper/2407.20955/) (2024)
- [Incorporating Structure and Chord Constraints in Symbolic Transformer-based Melodic Harmonization](https://mlsanity.com/paper/2512.07627/) (2025)
- [Pay (Cross) Attention to the Melody: Curriculum Masking for Single-Encoder Melodic Harmonization](https://mlsanity.com/paper/2601.16150/) (2026)
- [Structure-Aware Piano Accompaniment via Style Planning and Dataset-Aligned Pattern Retrieval](https://mlsanity.com/paper/2602.15074/) (2026)

## Abstract (arXiv)

Emotion-driven melody harmonization aims to generate diverse harmonies for a single melody to convey desired emotions. Previous research found it hard to alter the perceived emotional valence of lead sheets only by harmonizing the same melody with different chords, which may be attributed to the constraints imposed by the melody itself and the limitation of existing music representation. In this paper, we propose a novel functional representation for symbolic music. This new method takes musical keys into account, recognizing their significant role in shaping music's emotional character through major-minor tonality. It also allows for melodic variation with respect to keys and addresses the problem of data scarcity for better emotion modeling. A Transformer is employed to harmonize key-adaptable melodies, allowing for keys determined in rule-based or model-based manner. Experimental results confirm the effectiveness of our new representation in generating key-aware harmonies, with objective and subjective evaluations affirming the potential of our approach to convey specific valence for versatile melody.