# Structured Multi-Track Accompaniment Arrangement via Style Prior Modelling

arXiv: https://arxiv.org/abs/2310.16334 · HTML: https://mlsanity.com/paper/2310.16334/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 52 references that are in the map.
- First posted on arXiv 25 Oct 2023.

## Builds on (most cited first, 30 of 37)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context](https://mlsanity.com/paper/1901.02860/) (2019)
- [Generating Diverse High-Fidelity Images with VQ-VAE-2](https://mlsanity.com/paper/1906.00446/) (2019)
- [Jukebox: A Generative Model for Music](https://mlsanity.com/paper/2005.00341/) (2020)
- [Simple and Controllable Music Generation](https://mlsanity.com/paper/2306.05284/) (2023)
- [MusicLM: Generating Music From Text](https://mlsanity.com/paper/2301.11325/) (2023)
- [Everybody Dance Now](https://mlsanity.com/paper/1808.07371/) (2018)
- [Enabling Factorized Piano Music Modeling and Generation with the MAESTRO Dataset](https://mlsanity.com/paper/1810.12247/) (2018)
- [Music Transformer](https://mlsanity.com/paper/1809.04281/) (2018)
- [Text-Free Prosody-Aware Generative Spoken Language Modeling](https://mlsanity.com/paper/2109.03264/) (2021)
- [Compound Word Transformer: Learning to Compose Full-Song Music over Dynamic Directed Hypergraphs](https://mlsanity.com/paper/2101.02402/) (2021)
- [Cutting Music Source Separation Some Slakh: A Dataset to Study the Impact of Training Data Quality and Quantity](https://mlsanity.com/paper/1909.08494/) (2019)
- [POP909: A Pop-song Dataset for Music Arrangement Generation](https://mlsanity.com/paper/2008.07142/) (2020)
- [PopMAG: Pop Music Accompaniment Generation](https://mlsanity.com/paper/2008.07703/) (2020)
- [The Jazz Transformer on the Front Line: Exploring the Shortcomings of AI-composed Music through Quantitative Measures](https://mlsanity.com/paper/2008.01307/) (2020)
- [Music FaderNets: Controllable Music Generation Based On High-Level Features via Low-Level Feature Modelling](https://mlsanity.com/paper/2007.15474/) (2020)
- [Learning Interpretable Representation for Controllable Polyphonic Music Generation](https://mlsanity.com/paper/2008.07122/) (2020)
- [Anticipatory Music Transformer](https://mlsanity.com/paper/2306.08620/) (2023)
- [Polyffusion: A Diffusion Model for Polyphonic Score Generation with Internal and External Controls](https://mlsanity.com/paper/2307.10304/) (2023)
- [Improving Zero-shot Voice Style Transfer via Disentangled Representation Learning](https://mlsanity.com/paper/2103.09420/) (2021)
- [Deep Music Analogy Via Latent Representation Disentanglement](https://mlsanity.com/paper/1906.03626/) (2019)
- [Multitrack Music Transformer](https://mlsanity.com/paper/2207.06983/) (2022)
- [PIANOTREE VAE: Structured Representation Learning for Polyphonic Music](https://mlsanity.com/paper/2008.07118/) (2020)
- [AccoMontage: Accompaniment Arrangement via Phrase Selection and Style Transfer](https://mlsanity.com/paper/2108.11213/) (2021)
- [Automatic Analysis and Influence of Hierarchical Structure on Melody, Rhythm and Harmony in Popular Music](https://mlsanity.com/paper/2010.07518/) (2020)
- [Dance Style Transfer with Cross-modal Transformer](https://mlsanity.com/paper/2208.09406/) (2022)
- [A Unified Model for Zero-shot Music Source Separation, Transcription and Synthesis](https://mlsanity.com/paper/2108.03456/) (2021)

## Built on by (most cited first, 9 of 9)

- [Foundation Models for Music: A Survey](https://mlsanity.com/paper/2408.14340/) (2024)
- [Unifying Symbolic Music Arrangement: Track-Aware Reconstruction and Structured Tokenization](https://mlsanity.com/paper/2408.15176/) (2024)
- [ImprovNet -- Generating Controllable Musical Improvisations with Iterative Corruption Refinement](https://mlsanity.com/paper/2502.04522/) (2025)
- [BandCondiNet: Parallel Transformers-based Conditional Popular Music Generation with Multi-View Features](https://mlsanity.com/paper/2407.10462/) (2024)
- [PhraseVAE and PhraseLDM: Latent Diffusion for Full-Song Multitrack Symbolic Music Generation](https://mlsanity.com/paper/2512.11348/) (2025)
- [SymphonyGen: 3D Hierarchical Orchestral Generation with Controllable Harmony Skeleton](https://mlsanity.com/paper/2604.25498/) (2026)
- [Disentangling Score Content and Performance Style for Joint Piano Rendering and Transcription](https://mlsanity.com/paper/2509.23878/) (2025)
- [Structure-Aware Piano Accompaniment via Style Planning and Dataset-Aligned Pattern Retrieval](https://mlsanity.com/paper/2602.15074/) (2026)
- [Learning Music Style for Piano Arrangement Through Cross-Modal Bootstrapping](https://mlsanity.com/paper/2608.03050/) (2026)

## Abstract (arXiv)

In the realm of music AI, arranging rich and structured multi-track accompaniments from a simple lead sheet presents significant challenges. Such challenges include maintaining track cohesion, ensuring long-term coherence, and optimizing computational efficiency. In this paper, we introduce a novel system that leverages prior modelling over disentangled style factors to address these challenges. Our method presents a two-stage process: initially, a piano arrangement is derived from the lead sheet by retrieving piano texture styles; subsequently, a multi-track orchestration is generated by infusing orchestral function styles into the piano arrangement. Our key design is the use of vector quantization and a unique multi-stream Transformer to model the long-term flow of the orchestration style, which enables flexible, controllable, and structured music generation. Experiments show that by factorizing the arrangement task into interpretable sub-stages, our approach enhances generative capacity while improving efficiency. Additionally, our system supports a variety of music genres and provides style control at different composition hierarchies. We further show that our system achieves superior coherence, structure, and overall arrangement quality compared to existing baselines.