# XMusic: Towards a Generalized and Controllable Symbolic Music Generation Framework

arXiv: https://arxiv.org/abs/2501.08809 · HTML: https://mlsanity.com/paper/2501.08809/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jan 2025.

## Built on by (most cited first, 15 of 15)

- [A Comprehensive Survey on Generative AI for Video-to-Music Generation](https://mlsanity.com/paper/2502.12489/) (2025)
- [Vision-to-Music Generation: A Survey](https://mlsanity.com/paper/2503.21254/) (2025)
- [A Survey on Music Generation from Single-Modal, Cross-Modal, and Multi-Modal Perspectives](https://mlsanity.com/paper/2504.00837/) (2025)
- [Composer Vector: Style-steering Symbolic Music Generation in a Latent Space](https://mlsanity.com/paper/2604.03333/) (2026)
- [Video Soundtrack Generation by Aligning Emotions and Temporal Boundaries](https://mlsanity.com/paper/2502.10154/) (2025)
- [V2M-Zero: Zero-Pair Time-Aligned Video-to-Music Generation](https://mlsanity.com/paper/2603.11042/) (2026)
- [MusicAIR: A Multimodal AI Music Generation Framework Powered by an Algorithm-Driven Core](https://mlsanity.com/paper/2511.17323/) (2025)
- [Efficient Long-Sequence Diffusion Modeling for Symbolic Music Generation](https://mlsanity.com/paper/2603.00576/) (2026)
- [VidAudio-Bench: Benchmarking V2A and VT2A Generation across Four Audio Categories](https://mlsanity.com/paper/2604.10542/) (2026)
- [Latent Space Disentanglement via Activation Steering for Interpretable Attribute Control in Symbolic Music Generation](https://mlsanity.com/paper/2605.31295/) (2026)
- [Large Language Models' Internal Perception of Symbolic Music](https://mlsanity.com/paper/2507.12808/) (2025)
- [Amadeus: Autoregressive Model with Bidirectional Attribute Modelling for Symbolic Music](https://mlsanity.com/paper/2508.20665/) (2025)
- [Zero-Effort Image-to-Music Generation: An Interpretable RAG-based VLM Approach](https://mlsanity.com/paper/2509.22378/) (2025)
- [Explicit Tonal Tension Conditioning via Dual-Level Beam Search for Symbolic Music Generation](https://mlsanity.com/paper/2511.19342/) (2025)
- [Persian MusicGen: A Large-Scale Dataset and Culturally-Aware Generative Model for Persian Music](https://mlsanity.com/paper/2605.14765/) (2026)

## Abstract (arXiv)

In recent years, remarkable advancements in artificial intelligence-generated content (AIGC) have been achieved in the fields of image synthesis and text generation, generating content comparable to that produced by humans. However, the quality of AI-generated music has not yet reached this standard, primarily due to the challenge of effectively controlling musical emotions and ensuring high-quality outputs. This paper presents a generalized symbolic music generation framework, XMusic, which supports flexible prompts (i.e., images, videos, texts, tags, and humming) to generate emotionally controllable and high-quality symbolic music. XMusic consists of two core components, XProjector and XComposer. XProjector parses the prompts of various modalities into symbolic music elements (i.e., emotions, genres, rhythms and notes) within the projection space to generate matching music. XComposer contains a Generator and a Selector. The Generator generates emotionally controllable and melodious music based on our innovative symbolic music representation, whereas the Selector identifies high-quality symbolic music by constructing a multi-task learning scheme involving quality assessment, emotion recognition, and genre recognition tasks. In addition, we build XMIDI, a large-scale symbolic music dataset that contains 108,023 MIDI files annotated with precise emotion and genre labels. Objective and subjective evaluations show that XMusic significantly outperforms the current state-of-the-art methods with impressive music quality. Our XMusic has been awarded as one of the nine Highlights of Collectibles at WAIC 2023. The project homepage of XMusic is https://xmusic-project.github.io.