Structured Multi-Track Accompaniment Arrangement via Style Prior Modelling
Not on the map. Why?
Abstract (arXiv)
In the realm of music AI, arranging rich and structured multi-track accompaniments from a simple lead sheet presents significant challenges. Such challenges include maintaining track cohesion, ensuring long-term coherence, and optimizing computational efficiency. In this paper, we introduce a novel system that leverages prior modelling over disentangled style factors to address these challenges. Our method presents a two-stage process: initially, a piano arrangement is derived from the lead sheet by retrieving piano texture styles; subsequently, a multi-track orchestration is generated by infusing orchestral function styles into the piano arrangement. Our key design is the use of vector quantization and a unique multi-stream Transformer to model the long-term flow of the orchestration style, which enables flexible, controllable, and structured music generation. Experiments show that by factorizing the arrangement task into interpretable sub-stages, our approach enhances generative capacity while improving efficiency. Additionally, our system supports a variety of music genres and provides style control at different composition hierarchies. We further show that our system achieves superior coherence, structure, and overall arrangement quality compared to existing baselines.
Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 37 of its 52 references that are in the map. First posted on arXiv 25 Oct 2023.
Builds on
37 of 52 references.
References · 37
- 2017Attention Is All You Needcited by 59,932
- 2017Neural Discrete Representation Learningcited by 4,650
- 2016Gaussian Error Linear Units (GELUs)cited by 3,431
- 2019The Curious Case of Neural Text Degenerationcited by 2,881
- 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
- 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
- 2020Jukebox: A Generative Model for Musiccited by 535
- 2023Simple and Controllable Music Generationcited by 480
- 2023MusicLM: Generating Music From Textcited by 442
- 2018Everybody Dance Nowcited by 313
- 2018Enabling Factorized Piano Music Modeling and Generation with the MAESTRO Datasetcited by 215
- 2018Music Transformercited by 207
25 more
- 2021Text-Free Prosody-Aware Generative Spoken Language Modelingcited by 119
- 2021Compound Word Transformer: Learning to Compose Full-Song Music over Dynamic Directed Hypergraphscited by 109
- 2019Cutting Music Source Separation Some Slakh: A Dataset to Study the Impact of Training Data Quality and Quantitycited by 86
- 2020POP909: A Pop-song Dataset for Music Arrangement Generationcited by 83
- 2020PopMAG: Pop Music Accompaniment Generationcited by 68
- 2020The Jazz Transformer on the Front Line: Exploring the Shortcomings of AI-composed Music through Quantitative Measurescited by 53
- 2020Music FaderNets: Controllable Music Generation Based On High-Level Features via Low-Level Feature Modellingcited by 45
- 2020Learning Interpretable Representation for Controllable Polyphonic Music Generationcited by 40
- 2023Anticipatory Music Transformercited by 35
- 2023Polyffusion: A Diffusion Model for Polyphonic Score Generation with Internal and External Controlscited by 35
- 2021Improving Zero-shot Voice Style Transfer via Disentangled Representation Learningcited by 32
- 2019Deep Music Analogy Via Latent Representation Disentanglementcited by 31
- 2022Multitrack Music Transformercited by 28
- 2020PIANOTREE VAE: Structured Representation Learning for Polyphonic Musiccited by 25
- 2021AccoMontage: Accompaniment Arrangement via Phrase Selection and Style Transfercited by 22
- 2020Automatic Analysis and Influence of Hierarchical Structure on Melody, Rhythm and Harmony in Popular Musiccited by 21
- 2022Dance Style Transfer with Cross-modal Transformercited by 20
- 2021A Unified Model for Zero-shot Music Source Separation, Transcription and Synthesiscited by 18
- 2023Discrete Diffusion Probabilistic Models for Symbolic Music Generationcited by 16
- 2023CLaMP: Contrastive Language-Music Pre-training for Cross-Modal Symbolic Music Information Retrievalcited by 14
- 2023GETMusic: Generating Any Music Tracks with a Unified Representation and Diffusion Frameworkcited by 13
- 2022AccoMontage2: A Complete Harmonization and Accompaniment Arrangement Systemcited by 8
- 2023Q&A: Query-Based Representation Learning for Multi-Track Symbolic Music re-Arrangementcited by 7
- 2021Audio-to-symbolic Arrangement via Cross-modal Music Representation Learningcited by 6
- 2021Towards Automatic Instrumentation by Learning to Separate Parts in Symbolic Multitrack Musiccited by 5
Built on by
Citing papers per year: 2024: 3; 2025: 3; 2026: 3.
9 papers.
- 2024Foundation Models for Music: A Surveycited by 34
- 2024Unifying Symbolic Music Arrangement: Track-Aware Reconstruction and Structured Tokenizationcited by 3
- 2025ImprovNet -- Generating Controllable Musical Improvisations with Iterative Corruption Refinementcited by 3
- 2024BandCondiNet: Parallel Transformers-based Conditional Popular Music Generation with Multi-View Featurescited by 2
- 2025PhraseVAE and PhraseLDM: Latent Diffusion for Full-Song Multitrack Symbolic Music Generationcited by 2
- 2026SymphonyGen: 3D Hierarchical Orchestral Generation with Controllable Harmony Skeletoncited by 2
- 2025Disentangling Score Content and Performance Style for Joint Piano Rendering and Transcriptioncited by 0
- 2026Structure-Aware Piano Accompaniment via Style Planning and Dataset-Aligned Pattern Retrievalcited by 0
- 2026Learning Music Style for Piano Arrangement Through Cross-Modal Bootstrappingcited by 0
- 2026Learning Music Style for Piano Arrangement Through Cross-Modal Bootstrapping4 Aug 2026
- 2026SymphonyGen: 3D Hierarchical Orchestral Generation with Controllable Harmony Skeleton28 Apr 2026
- 2026Structure-Aware Piano Accompaniment via Style Planning and Dataset-Aligned Pattern Retrieval16 Feb 2026
- 2025PhraseVAE and PhraseLDM: Latent Diffusion for Full-Song Multitrack Symbolic Music Generation12 Dec 2025
- 2025Disentangling Score Content and Performance Style for Joint Piano Rendering and Transcription28 Sep 2025
- 2025ImprovNet -- Generating Controllable Musical Improvisations with Iterative Corruption Refinement6 Feb 2025
- 2024Unifying Symbolic Music Arrangement: Track-Aware Reconstruction and Structured Tokenization27 Aug 2024
- 2024Foundation Models for Music: A Survey26 Aug 2024
- 2024BandCondiNet: Parallel Transformers-based Conditional Popular Music Generation with Multi-View Features15 Jul 2024