# ByteComposer: a Human-like Melody Composition Method based on Language Model Agent

arXiv: https://arxiv.org/abs/2402.17785 · HTML: https://mlsanity.com/paper/2402.17785/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 24 references that are in the map.
- First posted on arXiv 24 Feb 2024.

## Builds on (most cited first, 14 of 14)

- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [Reflexion: Language Agents with Verbal Reinforcement Learning](https://mlsanity.com/paper/2303.11366/) (2023)
- [Toolformer: Language Models Can Teach Themselves to Use Tools](https://mlsanity.com/paper/2302.04761/) (2023)
- [Deep reinforcement learning from human preferences](https://mlsanity.com/paper/1706.03741/) (2017)
- [Self-Refine: Iterative Refinement with Self-Feedback](https://mlsanity.com/paper/2303.17651/) (2023)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [Sparks of Artificial General Intelligence: Early experiments with GPT-4](https://mlsanity.com/paper/2303.12712/) (2023)
- [ChemCrow: Augmenting large-language models with chemistry tools](https://mlsanity.com/paper/2304.05376/) (2023)
- [Make-An-Audio: Text-To-Audio Generation with Prompt-Enhanced Diffusion Models](https://mlsanity.com/paper/2301.12661/) (2023)
- [REFINER: Reasoning Feedback on Intermediate Representations](https://mlsanity.com/paper/2304.01904/) (2023)
- [Noise2Music: Text-conditioned Music Generation with Diffusion Models](https://mlsanity.com/paper/2302.03917/) (2023)
- [Language Models are General-Purpose Interfaces](https://mlsanity.com/paper/2206.06336/) (2022)
- [Mousai: Text-to-Music Generation with Long-Context Latent Diffusion](https://mlsanity.com/paper/2301.11757/) (2023)
- [MuseCoco: Generating Symbolic Music from Text](https://mlsanity.com/paper/2306.00110/) (2023)

## Built on by (most cited first, 6 of 6)

- [ComposerX: Multi-Agent Symbolic Music Composition with LLMs](https://mlsanity.com/paper/2404.18081/) (2024)
- [AI as Humanity's Salieri: Quantifying Linguistic Creativity of Language Models via Systematic Attribution of Machine Text against Web Text](https://mlsanity.com/paper/2410.04265/) (2024)
- [Foundation Models for Music: A Survey](https://mlsanity.com/paper/2408.14340/) (2024)
- [Can LLMs "Reason" in Music? An Evaluation of LLMs' Capability of Music Understanding and Generation](https://mlsanity.com/paper/2407.21531/) (2024)
- [Via Score to Performance: Efficient Human-Controllable Long Song Generation with Bar-Level Symbolic Notation](https://mlsanity.com/paper/2508.01394/) (2025)
- [BREATH: A Bio-Radar Embodied Agent for Tonal and Human-Aware Diffusion Music Generation](https://mlsanity.com/paper/2510.15895/) (2025)

## Abstract (arXiv)

Large Language Models (LLM) have shown encouraging progress in multimodal understanding and generation tasks. However, how to design a human-aligned and interpretable melody composition system is still under-explored. To solve this problem, we propose ByteComposer, an agent framework emulating a human's creative pipeline in four separate steps : "Conception Analysis - Draft Composition - Self-Evaluation and Modification - Aesthetic Selection". This framework seamlessly blends the interactive and knowledge-understanding features of LLMs with existing symbolic music generation models, thereby achieving a melody composition agent comparable to human creators. We conduct extensive experiments on GPT4 and several open-source large language models, which substantiate our framework's effectiveness. Furthermore, professional music composers were engaged in multi-dimensional evaluations, the final results demonstrated that across various facets of music composition, ByteComposer agent attains the level of a novice melody composer.