Agent-Driven Large Language Models for Mandarin Lyric Generation
On the map: LLM agents and tool use → Multimodal and embodied agents
Abstract (arXiv)
Generative Large Language Models have shown impressive in-context learning abilities, performing well across various tasks with just a prompt. Previous melody-to-lyric research has been limited by scarce high-quality aligned data and unclear standard for creativeness. Most efforts focused on general themes or emotions, which are less valuable given current language model capabilities. In tonal contour languages like Mandarin, pitch contours are influenced by both melody and tone, leading to variations in lyric-melody fit. Our study, validated by the Mpop600 dataset, confirms that lyricists and melody writers consider this fit during their composition process. In this research, we developed a multi-agent system that decomposes the melody-to-lyric task into sub-tasks, with each agent controlling rhyme, syllable count, lyric-melody alignment, and consistency. Listening tests were conducted via a diffusion-based singing voice synthesizer to evaluate the quality of lyrics generated by different agent groups.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use. Builds on 11 of its 30 references that are in the map. First posted on arXiv 2 Oct 2024.
Builds on
11 of 30 references.
References · 11
- 2020Language Models are Few-Shot Learnerscited by 32,199
- 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
- 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
- 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
- 2023Encouraging Divergent Thinking in Large Language Models through Multi-Agent Debatecited by 995
- 2023Exchange-of-Thought: Enhancing Large Language Model Capabilities through Cross-Model Communicationcited by 78
- 2020SongMASS: Automatic Song Writing with Pre-training and Alignment Constraintcited by 41
- 2021Learning to Generate Music With Sentimentcited by 29
- 2020Melody-Conditioned Lyrics Generation with SeqGANscited by 14
- 2017Lyrics-Based Music Genre Classification Using a Hierarchical Attention Networkcited by 7
- 2022Mandarin Singing Voice Synthesis with Denoising Diffusion Probabilistic Wasserstein GANcited by 5
Built on by
Citing papers per year: 2025: 1.
1 paper.
Similar, not cited
Close in wording, no citation either way
- 2023In BLOOM: Creativity and Affinity in Artificial Lyrics and Art
- 2026ARIA: Audio-Driven Melody-Tone Relation Modeling for Cantonese Lyric Authoring
- 2026Aligning Language Models for Lyric-to-Melody Generation with Rule-Based Musical Constraints
- 2025DiffRhythm 2: Efficient and High Fidelity Song Generation via Block Flow Matching
- 2024ByteComposer: a Human-like Melody Composition Method based on Language Model Agent
- 2024SongGLM: Lyric-to-Melody Generation with 2D Alignment Encoding and Multi-Task Pre-Training
- 2025YingMusic-Singer: Zero-shot Singing Voice Synthesis and Editing with Annotation-free Melody Guidance
- 2026Libretto: Giving LLM Agents a Sense of Musical Structure