TOMI: Transforming and Organizing Music Ideas for Multi-Track Compositions with Full-Song Structure
Not on the map. Why?
Abstract (arXiv)
Hierarchical planning is a powerful approach to model long sequences structurally. Aside from considering hierarchies in the temporal structure of music, this paper explores an even more important aspect: concept hierarchy, which involves generating music ideas, transforming them, and ultimately organizing them--across musical time and space--into a complete composition. To this end, we introduce TOMI (Transforming and Organizing Music Ideas) as a novel approach in deep music generation and develop a TOMI-based model via instruction-tuned foundation LLM. Formally, we represent a multi-track composition process via a sparse, four-dimensional space characterized by clips (short audio or MIDI segments), sections (temporal positions), tracks (instrument layers), and transformations (elaboration methods). Our model is capable of generating multi-track electronic music with full-song structure, and we further integrate the TOMI-based model with the REAPER digital audio workstation, enabling interactive human-AI co-creation. Experimental results demonstrate that our approach produces higher-quality electronic music with stronger structural coherence compared to baselines.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 27 of its 39 references that are in the map. First posted on arXiv 29 Jun 2025.
Builds on
27 of 39 references.
References · 27
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
- 2024GPT-4o System Cardcited by 4,588
- 2022Large-scale Contrastive Language-Audio Pretraining with Feature Fusion and Keyword-to-Caption Augmentationcited by 719
- 2016CNN Architectures for Large-Scale Audio Classificationcited by 704
- 2020Jukebox: A Generative Model for Musiccited by 542
- 2023Simple and Controllable Music Generationcited by 488
- 2023MusicLM: Generating Music From Textcited by 448
- 2024Audio Flamingo: A Novel Audio Language Model with Few-Shot Learning and Dialogue Abilitiescited by 186
- 2023MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Trainingcited by 181
- 2018A Hierarchical Latent Vector Model for Learning Long-Term Structure in Musiccited by 131
- 2016DeepBach: a Steerable Model for Bach Chorales Generationcited by 91
- 2023Music Understanding LLaMA: Advancing Text-to-Music Generation with Question Answering and Captioningcited by 77
15 more
- 2025YuE: Scaling Open Foundation Models for Long-Form Music Generationcited by 71
- 2024ChatMusician: Understanding and Generating Music Intrinsically with LLMcited by 68
- 2024Symbolic Music Generation with Non-Differentiable Rule Guided Diffusioncited by 54
- 2023MuseCoco: Generating Symbolic Music from Textcited by 52
- 2023Anticipatory Music Transformercited by 36
- 2023Polyffusion: A Diffusion Model for Polyphonic Score Generation with Internal and External Controlscited by 35
- 2024MuPT: A Generative Symbolic Music Pretrained Transformercited by 33
- 2021Controllable deep melody generation via hierarchical music structure representationcited by 30
- 2024Whole-Song Hierarchical Generation of Symbolic Music Using Cascaded Diffusion Modelscited by 25
- 2021MELONS: generating melody with long-term structure using transformers and structure graphcited by 18
- 2022What is missing in deep music generation? A study of repetition and structure in popular musiccited by 11
- 2023Composer's Assistant: An Interactive Transformer for Multi-Track MIDI Infillingcited by 9
- 2024Composer's Assistant 2: Interactive Multi-Track MIDI Infilling with Fine-Grained User Controlcited by 5
- 2024SymPAC: Scalable Symbolic Music Generation With Prompts And Constraintscited by 5
- 2018The Effect of Explicit Structure Encoding of Deep Neural Networks for Symbolic Music Generationcited by 2
Built on by
Not cited yet.