# MusicGen-Stem: Multi-stem music generation and edition through autoregressive modeling

arXiv: https://arxiv.org/abs/2501.01757 · HTML: https://mlsanity.com/paper/2501.01757/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Jan 2025.

## Built on by (most cited first, 12 of 12)

- [SongGen: A Single Stage Auto-regressive Transformer for Text-to-Song Generation](https://mlsanity.com/paper/2502.13128/) (2025)
- [MR-FlowDPO: Multi-Reward Direct Preference Optimization for Flow-Matching Text-to-Music Generation](https://mlsanity.com/paper/2512.10264/) (2025)
- [MGE-LDM: Joint Latent Diffusion for Simultaneous Music Generation and Source Extraction](https://mlsanity.com/paper/2505.23305/) (2025)
- [DARC: Drum accompaniment generation with fine-grained rhythm control](https://mlsanity.com/paper/2601.02357/) (2026)
- [Towards Real-Time Human-AI Musical Co-Performance: Accompaniment Generation with Latent Diffusion Models and MAX/MSP](https://mlsanity.com/paper/2604.07612/) (2026)
- [AI-Assisted Music Production: A User Study on Text-to-Music Models](https://mlsanity.com/paper/2509.23364/) (2025)
- [Stemphonic: All-at-once Flexible Multi-stem Music Generation](https://mlsanity.com/paper/2602.09891/) (2026)
- [Learning Relationships Between Separate Audio Tracks for Creative Applications](https://mlsanity.com/paper/2509.25296/) (2025)
- [MusRec: Zero-Shot Text-to-Music Editing via Rectified Flow and Diffusion Transformers](https://mlsanity.com/paper/2511.04376/) (2025)
- [FlowSonic: Stable Zero-Shot Music Editing via High-Order Trajectory Integration](https://mlsanity.com/paper/2607.17526/) (2026)
- [Iterative Audio Separation with Mixture Consistency via MIMO Model Extension](https://mlsanity.com/paper/2609.07226/) (2026)
- [Soundwich: Video Generation with Layered and Controllable Audio](https://mlsanity.com/paper/2610.00691/) (2026)

## Abstract (arXiv)

While most music generation models generate a mixture of stems (in mono or stereo), we propose to train a multi-stem generative model with 3 stems (bass, drums and other) that learn the musical dependencies between them. To do so, we train one specialized compression algorithm per stem to tokenize the music into parallel streams of tokens. Then, we leverage recent improvements in the task of music source separation to train a multi-stream text-to-music language model on a large dataset. Finally, thanks to a particular conditioning method, our model is able to edit bass, drums or other stems on existing or generated songs as well as doing iterative composition (e.g. generating bass on top of existing drums). This gives more flexibility in music generation algorithms and it is to the best of our knowledge the first open-source multi-stem autoregressive music generation model that can perform good quality generation and coherent source editing. Code and model weights will be released and samples are available on https://simonrouard.github.io/musicgenstem/.