mlsanityData to 7 Oct 2026

Structured Multi-Track Accompaniment Arrangement via Style Prior Modelling

arXiv 2310.16334 ↗First posted 25 Oct 2023cs.SD cs.AI cs.MMcited by 9

Not on the map. Why?

Abstract (arXiv)

In the realm of music AI, arranging rich and structured multi-track accompaniments from a simple lead sheet presents significant challenges. Such challenges include maintaining track cohesion, ensuring long-term coherence, and optimizing computational efficiency. In this paper, we introduce a novel system that leverages prior modelling over disentangled style factors to address these challenges. Our method presents a two-stage process: initially, a piano arrangement is derived from the lead sheet by retrieving piano texture styles; subsequently, a multi-track orchestration is generated by infusing orchestral function styles into the piano arrangement. Our key design is the use of vector quantization and a unique multi-stream Transformer to model the long-term flow of the orchestration style, which enables flexible, controllable, and structured music generation. Experiments show that by factorizing the arrangement task into interpretable sub-stages, our approach enhances generative capacity while improving efficiency. Additionally, our system supports a variety of music genres and provides style control at different composition hierarchies. We further show that our system achieves superior coherence, structure, and overall arrangement quality compared to existing baselines.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 37 of its 52 references that are in the map. First posted on arXiv 25 Oct 2023.

Builds on

37 of 52 references.

References · 37

  1. 2017Attention Is All You Needcited by 59,932
  2. 2017Neural Discrete Representation Learningcited by 4,650
  3. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  4. 2019The Curious Case of Neural Text Degenerationcited by 2,881
  5. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
  6. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  7. 2020Jukebox: A Generative Model for Musiccited by 535
  8. 2023Simple and Controllable Music Generationcited by 480
  9. 2023MusicLM: Generating Music From Textcited by 442
  10. 2018Everybody Dance Nowcited by 313
  11. 2018Enabling Factorized Piano Music Modeling and Generation with the MAESTRO Datasetcited by 215
  12. 2018Music Transformercited by 207
25 more
  1. 2021Text-Free Prosody-Aware Generative Spoken Language Modelingcited by 119
  2. 2021Compound Word Transformer: Learning to Compose Full-Song Music over Dynamic Directed Hypergraphscited by 109
  3. 2019Cutting Music Source Separation Some Slakh: A Dataset to Study the Impact of Training Data Quality and Quantitycited by 86
  4. 2020POP909: A Pop-song Dataset for Music Arrangement Generationcited by 83
  5. 2020PopMAG: Pop Music Accompaniment Generationcited by 68
  6. 2020The Jazz Transformer on the Front Line: Exploring the Shortcomings of AI-composed Music through Quantitative Measurescited by 53
  7. 2020Music FaderNets: Controllable Music Generation Based On High-Level Features via Low-Level Feature Modellingcited by 45
  8. 2020Learning Interpretable Representation for Controllable Polyphonic Music Generationcited by 40
  9. 2023Anticipatory Music Transformercited by 35
  10. 2023Polyffusion: A Diffusion Model for Polyphonic Score Generation with Internal and External Controlscited by 35
  11. 2021Improving Zero-shot Voice Style Transfer via Disentangled Representation Learningcited by 32
  12. 2019Deep Music Analogy Via Latent Representation Disentanglementcited by 31
  13. 2022Multitrack Music Transformercited by 28
  14. 2020PIANOTREE VAE: Structured Representation Learning for Polyphonic Musiccited by 25
  15. 2021AccoMontage: Accompaniment Arrangement via Phrase Selection and Style Transfercited by 22
  16. 2020Automatic Analysis and Influence of Hierarchical Structure on Melody, Rhythm and Harmony in Popular Musiccited by 21
  17. 2022Dance Style Transfer with Cross-modal Transformercited by 20
  18. 2021A Unified Model for Zero-shot Music Source Separation, Transcription and Synthesiscited by 18
  19. 2023Discrete Diffusion Probabilistic Models for Symbolic Music Generationcited by 16
  20. 2023CLaMP: Contrastive Language-Music Pre-training for Cross-Modal Symbolic Music Information Retrievalcited by 14
  21. 2023GETMusic: Generating Any Music Tracks with a Unified Representation and Diffusion Frameworkcited by 13
  22. 2022AccoMontage2: A Complete Harmonization and Accompaniment Arrangement Systemcited by 8
  23. 2023Q&A: Query-Based Representation Learning for Multi-Track Symbolic Music re-Arrangementcited by 7
  24. 2021Audio-to-symbolic Arrangement via Cross-modal Music Representation Learningcited by 6
  25. 2021Towards Automatic Instrumentation by Learning to Separate Parts in Symbolic Multitrack Musiccited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 3; 2026: 3.

9 papers.