# ReLyMe: Improving Lyric-to-Melody Generation by Incorporating Lyric-Melody Relationships

arXiv: https://arxiv.org/abs/2207.05688 · HTML: https://mlsanity.com/paper/2207.05688/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Jul 2022.

## Built on by (most cited first, 13 of 13)

- [MusicAgent: An AI Agent for Music Understanding and Generation with Large Language Models](https://mlsanity.com/paper/2310.11954/) (2023)
- [SongComposer: A Large Language Model for Lyric and Melody Generation in Song Composition](https://mlsanity.com/paper/2402.17645/) (2024)
- [Unsupervised Melody-to-Lyric Generation](https://mlsanity.com/paper/2305.19228/) (2023)
- [MeloForm: Generating Melody with Musical Form based on Expert Systems and Neural Networks](https://mlsanity.com/paper/2208.14345/) (2022)
- [A Survey on Music Generation from Single-Modal, Cross-Modal, and Multi-Modal Perspectives](https://mlsanity.com/paper/2504.00837/) (2025)
- [Syllable-level lyrics generation from melody exploiting character-level language model](https://mlsanity.com/paper/2310.00863/) (2023)
- [REFFLY: Melody-Constrained Lyrics Editing Model](https://mlsanity.com/paper/2409.00292/) (2024)
- [Relationships between Keywords and Strong Beats in Lyrical Music](https://mlsanity.com/paper/2412.04202/) (2024)
- [ALCAP: Alignment-Augmented Music Captioner](https://mlsanity.com/paper/2212.10901/) (2022)
- [MusicAIR: A Multimodal AI Music Generation Framework Powered by an Algorithm-Driven Core](https://mlsanity.com/paper/2511.17323/) (2025)
- [Unsupervised Melody-Guided Lyrics Generation](https://mlsanity.com/paper/2305.07760/) (2023)
- [SongSong: A Time Phonograph for Chinese SongCi Music from Thousand of Years Away](https://mlsanity.com/paper/2602.24071/) (2026)
- [Aligning Language Models for Lyric-to-Melody Generation with Rule-Based Musical Constraints](https://mlsanity.com/paper/2604.18489/) (2026)

## Abstract (arXiv)

Lyric-to-melody generation, which generates melody according to given lyrics, is one of the most important automatic music composition tasks. With the rapid development of deep learning, previous works address this task with end-to-end neural network models. However, deep learning models cannot well capture the strict but subtle relationships between lyrics and melodies, which compromises the harmony between lyrics and generated melodies. In this paper, we propose ReLyMe, a method that incorporates Relationships between Lyrics and Melodies from music theory to ensure the harmony between lyrics and melodies. Specifically, we first introduce several principles that lyrics and melodies should follow in terms of tone, rhythm, and structure relationships. These principles are then integrated into neural network lyric-to-melody models by adding corresponding constraints during the decoding process to improve the harmony between lyrics and melodies. We use a series of objective and subjective metrics to evaluate the generated melodies. Experiments on both English and Chinese song datasets show the effectiveness of ReLyMe, demonstrating the superiority of incorporating lyric-melody relationships from the music domain into neural lyric-to-melody generation.