# Music Source Separation with Band-Split RoPE Transformer

arXiv: https://arxiv.org/abs/2309.02612 · HTML: https://mlsanity.com/paper/2309.02612/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Sep 2023.

## Built on by (most cited first, 30 of 37)

- [Beat this! Accurate beat tracking without DBN postprocessing](https://mlsanity.com/paper/2407.21658/) (2024)
- [Scoring Time Intervals using Non-Hierarchical Transformer For Automatic Piano Transcription](https://mlsanity.com/paper/2404.09466/) (2024)
- [A Stem-Agnostic Single-Decoder System for Music Source Separation Beyond Four Stems](https://mlsanity.com/paper/2406.18747/) (2024)
- [Music Foundation Model as Generic Booster for Music Downstream Tasks](https://mlsanity.com/paper/2411.01135/) (2024)
- [Music Source Restoration](https://mlsanity.com/paper/2505.21827/) (2025)
- [Pushing the Frontier of Full-Song Generation: Hierarchical Autoregressive Planning Meets Flow-Matching Rendering](https://mlsanity.com/paper/2607.20253/) (2026)
- [A Two-Stage Band-Split Mamba-2 Network For Music Separation](https://mlsanity.com/paper/2409.06245/) (2024)
- [User-guided Generative Source Separation](https://mlsanity.com/paper/2507.01339/) (2025)
- [AnyAccomp: Generalizable Accompaniment Generation via Quantized Melodic Bottleneck](https://mlsanity.com/paper/2509.14052/) (2025)
- [Efficient and Fast Generative-Based Singing Voice Separation using a Latent Diffusion Model](https://mlsanity.com/paper/2511.20470/) (2025)
- [An Ensemble Approach to Music Source Separation: A Comparative Analysis of Conventional and Hierarchical Stem Separation](https://mlsanity.com/paper/2410.20773/) (2024)
- [Improving Music Source Separation with Diffusion and Consistency Refinement](https://mlsanity.com/paper/2412.06965/) (2024)
- [Training-Free Multi-Step Audio Source Separation](https://mlsanity.com/paper/2505.19534/) (2025)
- [The Spheres Dataset: Multitrack Orchestral Recordings for Music Source Separation and Information Retrieval](https://mlsanity.com/paper/2511.21247/) (2025)
- [Generating Separated Singing Vocals Using a Diffusion Model Conditioned on Music Mixtures](https://mlsanity.com/paper/2511.21342/) (2025)
- [Summary of The Inaugural Music Source Restoration Challenge](https://mlsanity.com/paper/2601.04343/) (2026)
- [MoXaRt: Audio-Visual Object-Guided Sound Interaction for XR](https://mlsanity.com/paper/2603.10465/) (2026)
- [STEB: A Speech-to-Speech Translation Expressiveness Benchmark for Evaluating Beyond Translation Fidelity](https://mlsanity.com/paper/2606.25529/) (2026)
- [Solving Copyright Infringement on Short Video Platforms: Novel Datasets and an Audio Restoration Deep Learning Pipeline](https://mlsanity.com/paper/2504.21772/) (2025)
- [Source Separation of Small Classical Ensembles: Challenges and Opportunities](https://mlsanity.com/paper/2505.17823/) (2025)
- [Mamba2 Meets Silence: Robust Vocal Source Separation for Sparse Regions](https://mlsanity.com/paper/2508.14556/) (2025)
- [TISDiSS: A Training-Time and Inference-Time Scalable Framework for Discriminative Source Separation](https://mlsanity.com/paper/2509.15666/) (2025)
- [DuoTok: Source-Aware Dual-Track Music Tokenization for Vocal-Accompaniment Generation](https://mlsanity.com/paper/2511.20224/) (2025)
- [A Conditioned UNet for Music Source Separation](https://mlsanity.com/paper/2512.15532/) (2025)
- [BASS: Benchmarking Audio LMs for Musical Structure and Semantic Reasoning](https://mlsanity.com/paper/2602.04085/) (2026)
- [Tutti: Expressive Multi-Singer Synthesis via Structure-Level Timbre Control and Vocal Texture Modeling](https://mlsanity.com/paper/2602.08233/) (2026)
- [Multi-Stage Music Source Restoration with BandSplit-RoFormer Separation and HiFi++ GAN](https://mlsanity.com/paper/2603.04032/) (2026)
- [The Costs of Reproducibility in Music Separation Research: a Replication of Band-Split RNN](https://mlsanity.com/paper/2603.09187/) (2026)
- [Music Source Restoration with Ensemble Separation and Targeted Reconstruction](https://mlsanity.com/paper/2603.16926/) (2026)
- [Diff-VS: Efficient Audio-Aware Diffusion U-Net for Vocals Separation](https://mlsanity.com/paper/2604.01120/) (2026)

## Abstract (arXiv)

Music source separation (MSS) aims to separate a music recording into multiple musically distinct stems, such as vocals, bass, drums, and more. Recently, deep learning approaches such as convolutional neural networks (CNNs) and recurrent neural networks (RNNs) have been used, but the improvement is still limited. In this paper, we propose a novel frequency-domain approach based on a Band-Split RoPE Transformer (called BS-RoFormer). BS-RoFormer relies on a band-split module to project the input complex spectrogram into subband-level representations, and then arranges a stack of hierarchical Transformers to model the inner-band as well as inter-band sequences for multi-band mask estimation. To facilitate training the model for MSS, we propose to use the Rotary Position Embedding (RoPE). The BS-RoFormer system trained on MUSDB18HQ and 500 extra songs ranked the first place in the MSS track of Sound Demixing Challenge (SDX23). Benchmarking a smaller version of BS-RoFormer on MUSDB18HQ, we achieve state-of-the-art result without extra training data, with 9.80 dB of average SDR.