mlsanityData to 14 Oct 2026

MuseCoco: Generating Symbolic Music from Text

arXiv 2306.00110 ↗First posted 31 May 2023cs.SD cs.AI cs.CLcited by 52

Not on the map. Why?

Abstract (arXiv)

Generating music from text descriptions is a user-friendly mode since the text is a relatively easy interface for user engagement. While some approaches utilize texts to control music audio generation, editing musical elements in generated audio is challenging for users. In contrast, symbolic music offers ease of editing, making it more accessible for users to manipulate specific musical elements. In this paper, we propose MuseCoco, which generates symbolic music from text descriptions with musical attributes as the bridge to break down the task into text-to-attribute understanding and attribute-to-music generation stages. MuseCoCo stands for Music Composition Copilot that empowers musicians to generate music directly from given text descriptions, offering a significant improvement in efficiency compared to creating music entirely from scratch. The system has two main advantages: Firstly, it is data efficient. In the attribute-to-music generation stage, the attributes can be directly extracted from music sequences, making the model training self-supervised. In the text-to-attribute understanding stage, the text is synthesized and refined by ChatGPT based on the defined attribute templates. Secondly, the system can achieve precise control with specific attributes in text descriptions and offers multiple control options through attribute-conditioned or text-conditioned approaches. MuseCoco outperforms baseline systems in terms of musicality, controllability, and overall score by at least 1.27, 1.08, and 1.32 respectively. Besides, there is a notable enhancement of about 20% in objective control accuracy. In addition, we have developed a robust large-scale model with 1.2 billion parameters, showcasing exceptional controllability and musicality.

Key facts. Cited by 52 papers in the map (data through 14 Oct 2026). Builds on 35 of its 44 references that are in the map. First posted on arXiv 31 May 2023.

Builds on

35 of 44 references.

References · 35

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  5. 2023GPT-4 Technical Reportcited by 18,396
  6. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  7. 2022Photorealistic Text-to-Image Diffusion Models with Deep Language Understandingcited by 6,355
  8. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  9. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  10. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  11. 2017FiLM: Visual Reasoning with a General Conditioning Layercited by 2,333
  12. 2021Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processingcited by 2,234
23 more
  1. 2020Transformers are RNNs: Fast Autoregressive Transformers with Linear Attentioncited by 2,022
  2. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 762
  3. 2023MusicLM: Generating Music From Textcited by 448
  4. 2023Noise2Music: Text-conditioned Music Generation with Diffusion Modelscited by 173
  5. 2021AdaSpeech: Adaptive Text to Speech for Custom Voicecited by 98
  6. 2021MusicBERT: Symbolic Music Understanding with Large-Scale Pre-Trainingcited by 89
  7. 2022MuLan: A Joint Embedding of Music Audio and Natural Languagecited by 87
  8. 2020POP909: A Pop-song Dataset for Music Arrangement Generationcited by 83
  9. 2021Video Background Music Generation with Controllable Music Transformercited by 56
  10. 2021EMOPIA: A Multi-Modal Pop Piano Dataset For Emotion Recognition and Emotion-based Music Generationcited by 53
  11. 2020MMM : Exploring Conditional Multi-Track Music Generation with the Transformercited by 49
  12. 2022Museformer: Transformer with Fine- and Coarse-Grained Attention for Music Generationcited by 49
  13. 2020Music FaderNets: Controllable Music Generation Based On High-Level Features via Low-Level Feature Modellingcited by 45
  14. 2020Learning Interpretable Representation for Controllable Polyphonic Music Generationcited by 40
  15. 2022Quantized GAN for Complex Music Generation from Dance Videoscited by 37
  16. 2019Encoding Musical Style with Transformer Autoencoderscited by 35
  17. 2022Symphony Generation with Permutation Invariant Language Modelcited by 35
  18. 2023ERNIE-Music: Text-to-Waveform Music Generation with Diffusion Modelscited by 15
  19. 2022Exploring the Efficacy of Pre-trained Checkpoints in Text-to-Music Generation Taskcited by 13
  20. 2018DeepJ: Style-Specific Music Generationcited by 12
  21. 2021Structure-Enhanced Pop Music Generation via Harmony-Aware Learningcited by 11
  22. 2022Controlling Perceived Emotion in Symbolic Music Generation with Monte Carlo Tree Searchcited by 11
  23. 2022Generating music with sentiment using Transformer-GANscited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 1; 2024: 20; 2025: 18; 2026: 13.

52 papers.

Sort:

By area:

  1. 2024ChatMusician: Understanding and Generating Music Intrinsically with LLMcited by 68
  2. 2024ComposerX: Multi-Agent Symbolic Music Composition with LLMscited by 42
  3. 2024Foundation Models for Music: A Surveycited by 37
  4. 2024MuPT: A Generative Symbolic Music Pretrained Transformercited by 33
  5. 2024MusicMagus: Zero-Shot Text-to-Music Editing via Diffusion Modelscited by 32
  6. 2024Next Token Prediction Towards Multimodal Intelligence: A Comprehensive Surveycited by 27
  7. 2023MusicAgent: An AI Agent for Music Understanding and Generation with Large Language Modelscited by 20
  8. 2024Text2midi: Generating Symbolic Music from Captionscited by 19
  9. 2025Learning Musical Representations for Music Performance Question Answeringcited by 17
  10. 2024MuChin: A Chinese Colloquial Description Benchmark for Evaluating Language Models in the Field of Musiccited by 16
  11. 2024Natural Language Processing Methods for Symbolic Music Generation and Information Retrieval: a Surveycited by 10
  12. 2024The Music Maestro or The Musically Challenged, A Massive Music Evaluation Benchmark for Large Language Modelscited by 10
38 more
  1. 2025Scaling Self-Supervised Representation Learning for Symbolic Piano Performancecited by 9
  2. 2024ByteComposer: a Human-like Melody Composition Method based on Language Model Agentcited by 6
  3. 2024A Survey of Deep Learning Audio Generation Methodscited by 5
  4. 2024Emotion-driven Piano Music Generation via Two-stage Disentanglement and Functional Representationcited by 5
  5. 2024Emotion-Driven Melody Harmonization via Melodic Variation and Functional Representationcited by 4
  6. 2025NOTA: Multimodal Music Notation Understanding for Visual Large Language Modelcited by 4
  7. 2025A Survey on Music Generation from Single-Modal, Cross-Modal, and Multi-Modal Perspectivescited by 4
  8. 2025Text2midi-InferAlign: Improving Symbolic Music Generation with Inference-Time Alignmentcited by 4
  9. 2025Music Audio-Visual Question Answering Requires Specialized Multimodal Designscited by 4
  10. 2024Unifying Symbolic Music Arrangement: Track-Aware Reconstruction and Structured Tokenizationcited by 3
  11. 2025QAMRO: Quality-aware Adaptive Margin Ranking Optimization for Human-aligned Assessment of Audio Generation Systemscited by 3
  12. 2026Composer Vector: Style-steering Symbolic Music Generation in a Latent Spacecited by 3
  13. 2024MuDiT & MuSiT: Alignment with Colloquial Expression in Description-to-Song Generationcited by 2
  14. 2024BandCondiNet: Parallel Transformers-based Conditional Popular Music Generation with Multi-View Featurescited by 2
  15. 2024Applications and Advances of Artificial Intelligence in Music Generation:A Reviewcited by 2
  16. 2025PhraseVAE and PhraseLDM: Latent Diffusion for Full-Song Multitrack Symbolic Music Generationcited by 2
  17. 2024Prevailing Research Areas for Music AI in the Era of Foundation Modelscited by 1
  18. 2025Genre Controlled Music Generation via Activation Steeringcited by 1
  19. 2025From Generality to Mastery: Composer-Style Symbolic Music Generation via Large-Scale Pre-trainingcited by 1
  20. 2025MIDI-LLM: Improving Text-to-MIDI Music Generation via Adapting Large Language Modelscited by 1
  21. 2026How Far Can Pretrained LLMs Go in Symbolic Music? Controlled Comparisons of Supervised and Preference-based Adaptationcited by 1
  22. 2026ViTex: Visual Texture Control for Multi-Track Symbolic Music Generation via Discrete Diffusion Modelscited by 1
  23. 2026BeatEdit: Symbolic Music Generation as Explicit Editingcited by 1
  24. 2024MuseBarControl: Enhancing Fine-Grained Control in Symbolic Music Generation through Pre-Training and Counterfactual Losscited by 0
  25. 2025TOMI: Transforming and Organizing Music Ideas for Multi-Track Compositions with Full-Song Structurecited by 0
  26. 2025MixAssist: An Audio-Language Dataset for Co-Creative AI Assistance in Music Mixingcited by 0
  27. 2025Large Language Models' Internal Perception of Symbolic Musiccited by 0
  28. 2025Amadeus: Autoregressive Model with Bidirectional Attribute Modelling for Symbolic Musiccited by 0
  29. 2025A Traditional Approach to Symbolic Piano Continuationcited by 0
  30. 2025SAGE-Music: Low-Latency Symbolic Music Generation via Attribute-Specialized Key-Value Head Sharingcited by 0
  31. 2025MusRec: Zero-Shot Text-to-Music Editing via Rectified Flow and Diffusion Transformerscited by 0
  32. 2026ONOTE: Hypergraph-Grounded Omnimodal Reasoning for Computational Music Sciencecited by 0
  33. 2026Verifier-Guided Twelve-Tone Composition: A Generate-Verify-Repair Harness for Symbolic Music Generationcited by 0
  34. 2026FlowSonic: Stable Zero-Shot Music Editing via High-Order Trajectory Integrationcited by 0
  35. 2026DDSynth-RL: Audio Synthesizer Inversion via Discrete Diffusion with Reinforcement Learningcited by 0
  36. 2026Learning Music Style for Piano Arrangement Through Cross-Modal Bootstrappingcited by 0
  37. 2026Agogic: Performance-Timed Music Tokens for LLM-Native Text-to-Symbolic-Music Generationcited by 0
  38. 2026Diff-Symbo: Text-Controlled Long-Duration Symbolic Music Generation Using Autoregressive Latent Diffusion Modelcited by 0

50 most cited of 52 All 52 (JSON)