# MMM : Exploring Conditional Multi-Track Music Generation with the Transformer

arXiv: https://arxiv.org/abs/2008.06048 · HTML: https://mlsanity.com/paper/2008.06048/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Aug 2020.

## Built on by (most cited first, 30 of 49)

- [Compound Word Transformer: Learning to Compose Full-Song Music over Dynamic Directed Hypergraphs](https://mlsanity.com/paper/2101.02402/) (2021)
- [MuseCoco: Generating Symbolic Music from Text](https://mlsanity.com/paper/2306.00110/) (2023)
- [Foundation Models for Music: A Survey](https://mlsanity.com/paper/2408.14340/) (2024)
- [Symphony Generation with Permutation Invariant Language Model](https://mlsanity.com/paper/2205.05448/) (2022)
- [Multitrack Music Transformer](https://mlsanity.com/paper/2207.06983/) (2022)
- [miditok: A Python package for MIDI file tokenization](https://mlsanity.com/paper/2310.17202/) (2023)
- [MusIAC: An extensible generative framework for Music Infilling Applications with multi-level Control](https://mlsanity.com/paper/2202.05528/) (2022)
- [JEN-1 Composer: A Unified Framework for High-Fidelity Multi-Track Music Generation](https://mlsanity.com/paper/2310.19180/) (2023)
- [GETMusic: Generating Any Music Tracks with a Unified Representation and Diffusion Framework](https://mlsanity.com/paper/2305.10841/) (2023)
- [Music Composition with Deep Learning: A Review](https://mlsanity.com/paper/2108.12290/) (2021)
- [Natural Language Processing Methods for Symbolic Music Generation and Information Retrieval: a Survey](https://mlsanity.com/paper/2402.17467/) (2024)
- [Composer's Assistant: An Interactive Transformer for Multi-Track MIDI Infilling](https://mlsanity.com/paper/2301.12525/) (2023)
- [ComMU: Dataset for Combinatorial Music Generation](https://mlsanity.com/paper/2211.09385/) (2022)
- [Score Transformer: Generating Musical Score from Note-level Representation](https://mlsanity.com/paper/2112.00355/) (2021)
- [YM2413-MDB: A Multi-Instrumental FM Video Game Music Dataset with Emotion Annotations](https://mlsanity.com/paper/2211.07131/) (2022)
- [The GigaMIDI Dataset with Features for Expressive Music Performance Detection](https://mlsanity.com/paper/2502.17726/) (2025)
- [Impact of time and note duration tokenizations on deep learning symbolic music modeling](https://mlsanity.com/paper/2310.08497/) (2023)
- [Towards Automatic Instrumentation by Learning to Separate Parts in Symbolic Multitrack Music](https://mlsanity.com/paper/2107.05916/) (2021)
- [Subjective Evaluation of Deep Learning Models for Symbolic Music Composition](https://mlsanity.com/paper/2203.14641/) (2022)
- [A Survey of Deep Learning Audio Generation Methods](https://mlsanity.com/paper/2406.00146/) (2024)
- [Composer's Assistant 2: Interactive Multi-Track MIDI Infilling with Fine-Grained User Control](https://mlsanity.com/paper/2407.14700/) (2024)
- [Multi-Track MusicLDM: Towards Versatile Music Generation with Latent Diffusion Model](https://mlsanity.com/paper/2409.02845/) (2024)
- [PerTok: Expressive Encoding and Modeling of Symbolic Musical Ideas and Variations](https://mlsanity.com/paper/2410.02060/) (2024)
- [Evaluating Human-AI Interaction via Usability, User Experience and Acceptance Measures for MMM-C: A Creative AI System for Music Composition](https://mlsanity.com/paper/2504.14071/) (2025)
- [Moonbeam: A MIDI Foundation Model Using Both Absolute and Relative Music Attributes](https://mlsanity.com/paper/2505.15559/) (2025)
- [A Survey on Artificial Intelligence for Music Generation: Agents, Domains and Perspectives](https://mlsanity.com/paper/2210.13944/) (2022)
- [Symbolic Music Structure Analysis with Graph Representations and Changepoint Detection Methods](https://mlsanity.com/paper/2303.13881/) (2023)
- [Unifying Symbolic Music Arrangement: Track-Aware Reconstruction and Structured Tokenization](https://mlsanity.com/paper/2408.15176/) (2024)
- [Exploring the Needs of Practising Musicians in Co-Creative AI Through Co-Design](https://mlsanity.com/paper/2502.09055/) (2025)
- [A Survey of Music Generation in the Context of Interaction](https://mlsanity.com/paper/2402.15294/) (2024)

## Abstract (arXiv)

We propose the Multi-Track Music Machine (MMM), a generative system based on the Transformer architecture that is capable of generating multi-track music. In contrast to previous work, which represents musical material as a single time-ordered sequence, where the musical events corresponding to different tracks are interleaved, we create a time-ordered sequence of musical events for each track and concatenate several tracks into a single sequence. This takes advantage of the Transformer's attention-mechanism, which can adeptly handle long-term dependencies. We explore how various representations can offer the user a high degree of control at generation time, providing an interactive demo that accommodates track-level and bar-level inpainting, and offers control over track instrumentation and note density.