# Neural Melody Composition from Lyrics

arXiv: https://arxiv.org/abs/1809.04318 · HTML: https://mlsanity.com/paper/1809.04318/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Sep 2018.

## Built on by (most cited first, 10 of 10)

- [TeleMelody: Lyric-to-Melody Generation with a Template-Based Two-Stage Method](https://mlsanity.com/paper/2109.09617/) (2021)
- [Automatic Song Translation for Tonal Languages](https://mlsanity.com/paper/2203.13420/) (2022)
- [SongComposer: A Large Language Model for Lyric and Melody Generation in Song Composition](https://mlsanity.com/paper/2402.17645/) (2024)
- [Re-creation of Creations: A New Paradigm for Lyric-to-Melody Generation](https://mlsanity.com/paper/2208.05697/) (2022)
- [Controllable Lyrics-to-Melody Generation](https://mlsanity.com/paper/2306.02613/) (2023)
- [Relationships between Keywords and Strong Beats in Lyrical Music](https://mlsanity.com/paper/2412.04202/) (2024)
- [MusicAIR: A Multimodal AI Music Generation Framework Powered by an Algorithm-Driven Core](https://mlsanity.com/paper/2511.17323/) (2025)
- [Deep Attention-Based Alignment Network for Melody Generation from Incomplete Lyrics](https://mlsanity.com/paper/2301.10015/) (2023)
- [Vocal Melody Construction for Persian Lyrics Using LSTM Recurrent Neural Networks](https://mlsanity.com/paper/2410.18203/) (2024)
- [SLEEPING-DISCO 9M: A large-scale pre-training dataset for generative music modeling](https://mlsanity.com/paper/2506.14293/) (2025)

## Abstract (arXiv)

In this paper, we study a novel task that learns to compose music from natural language. Given the lyrics as input, we propose a melody composition model that generates lyrics-conditional melody as well as the exact alignment between the generated melody and the given lyrics simultaneously. More specifically, we develop the melody composition model based on the sequence-to-sequence framework. It consists of two neural encoders to encode the current lyrics and the context melody respectively, and a hierarchical decoder to jointly produce musical notes and the corresponding alignment. Experimental results on lyrics-melody pairs of 18,451 pop songs demonstrate the effectiveness of our proposed methods. In addition, we apply a singing voice synthesizer software to synthesize the "singing" of the lyrics and melodies for human evaluation. Results indicate that our generated melodies are more melodious and tuneful compared with the baseline method.