# Deep Attention-Based Alignment Network for Melody Generation from Incomplete Lyrics

arXiv: https://arxiv.org/abs/2301.10015 · HTML: https://mlsanity.com/paper/2301.10015/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 19 references that are in the map.
- First posted on arXiv 23 Jan 2023.

## Builds on (most cited first, 12 of 12)

- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [MuseGAN: Multi-track Sequential Generative Adversarial Networks for Symbolic Music Generation and Accompaniment](https://mlsanity.com/paper/1709.06298/) (2017)
- [C-RNN-GAN: Continuous recurrent neural networks with adversarial training](https://mlsanity.com/paper/1611.09904/) (2016)
- [SongMASS: Automatic Song Writing with Pre-training and Alignment Constraint](https://mlsanity.com/paper/2012.05168/) (2020)
- [Conditional LSTM-GAN for Melody Generation from Lyrics](https://mlsanity.com/paper/1908.05551/) (2019)
- [Song From PI: A Musically Plausible Network for Pop Music Generation](https://mlsanity.com/paper/1611.03477/) (2016)
- [Neural Melody Composition from Lyrics](https://mlsanity.com/paper/1809.04318/) (2018)
- [Deep Cross-Modal Correlation Learning for Audio and Lyrics in Music Retrieval](https://mlsanity.com/paper/1711.08976/) (2017)
- [Algorithmic Songwriting with ALYSIA](https://mlsanity.com/paper/1612.01058/) (2016)
- [Interpretable Melody Generation from Lyrics with Discrete-Valued Adversarial Training](https://mlsanity.com/paper/2206.15027/) (2022)

## Abstract (arXiv)

We propose a deep attention-based alignment network, which aims to automatically predict lyrics and melody with given incomplete lyrics as input in a way similar to the music creation of humans. Most importantly, a deep neural lyrics-to-melody net is trained in an encoder-decoder way to predict possible pairs of lyrics-melody when given incomplete lyrics (few keywords). The attention mechanism is exploited to align the predicted lyrics with the melody during the lyrics-to-melody generation. The qualitative and quantitative evaluation metrics reveal that the proposed method is indeed capable of generating proper lyrics and corresponding melody for composing new songs given a piece of incomplete seed lyrics.