# Improving Real-Time Music Accompaniment Separation with MMDenseNet

arXiv: https://arxiv.org/abs/2407.00657 · HTML: https://mlsanity.com/paper/2407.00657/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 14 of its 23 references that are in the map.
- First posted on arXiv 30 Jun 2024.

## Builds on (most cited first, 14 of 14)

- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Axial-DeepLab: Stand-Alone Axial-Attention for Panoptic Segmentation](https://mlsanity.com/paper/2003.07853/) (2020)
- [Hybrid Transformers for Music Source Separation](https://mlsanity.com/paper/2211.08553/) (2022)
- [Pointwise Convolutional Neural Networks](https://mlsanity.com/paper/1712.05245/) (2017)
- [Memory Transformer](https://mlsanity.com/paper/2006.11527/) (2020)
- [Music Source Separation with Band-split RNN](https://mlsanity.com/paper/2209.15174/) (2022)
- [Decoupling Magnitude and Phase Estimation with Deep ResUNet for Music Source Separation](https://mlsanity.com/paper/2109.05418/) (2021)
- [TF-GridNet: Making Time-Frequency Domain Models Great Again for Monaural Speaker Separation](https://mlsanity.com/paper/2209.03952/) (2022)
- [Music Demixing Challenge 2021](https://mlsanity.com/paper/2108.13559/) (2021)
- [KUIELab-MDX-Net: A Two-Stream Neural Network for Music Demixing](https://mlsanity.com/paper/2111.12203/) (2021)
- [Multi-scale Multi-band DenseNets for Audio Source Separation](https://mlsanity.com/paper/1706.09588/) (2017)
- [Toward Universal Speech Enhancement for Diverse Input Conditions](https://mlsanity.com/paper/2309.17384/) (2023)
- [Voice and accompaniment separation in music using self-attention convolutional neural network](https://mlsanity.com/paper/2003.08954/) (2020)
- [Music-robust Automatic Lyrics Transcription of Polyphonic Music](https://mlsanity.com/paper/2204.03306/) (2022)

## Abstract (arXiv)

Music source separation aims to separate polyphonic music into different types of sources. Most existing methods focus on enhancing the quality of separated results by using a larger model structure, rendering them unsuitable for deployment on edge devices. Moreover, these methods may produce low-quality output when the input duration is short, making them impractical for real-time applications. Therefore, the goal of this paper is to enhance a lightweight model, MMDenstNet, to strike a balance between separation quality and latency for real-time applications. Different directions of improvement are explored or proposed in this paper, including complex ideal ratio mask, self-attention, band-merge-split method, and feature look back. Source-to-distortion ratio, real-time factor, and optimal latency are employed to evaluate the performance. To align with our application requirements, the evaluation process in this paper focuses on the separation performance of the accompaniment part. Experimental results demonstrate that our improvement achieves low real-time factor and optimal latency while maintaining acceptable separation quality.