# The Cocktail Fork Problem: Three-Stem Audio Separation for Real-World Soundtracks

arXiv: https://arxiv.org/abs/2110.09958 · HTML: https://mlsanity.com/paper/2110.09958/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 Oct 2021.

## Built on by (most cited first, 19 of 19)

- [MERT: Acoustic Music Understanding Model with Large-Scale Self-supervised Training](https://mlsanity.com/paper/2306.00107/) (2023)
- [TIGER: Time-frequency Interleaved Gain Extraction and Reconstruction for Efficient Speech Separation](https://mlsanity.com/paper/2410.01469/) (2024)
- [OmniChat: Enhancing Spoken Dialogue Systems with Scalable Synthetic Data for Diverse Scenarios](https://mlsanity.com/paper/2501.01384/) (2025)
- [A Generalized Bandsplit Neural Network for Cinematic Audio Source Separation](https://mlsanity.com/paper/2309.02539/) (2023)
- [Self-Supervised Audio-Visual Soundscape Stylization](https://mlsanity.com/paper/2409.14340/) (2024)
- [A Stem-Agnostic Single-Decoder System for Music Source Separation Beyond Four Stems](https://mlsanity.com/paper/2406.18747/) (2024)
- [Benchmarks and leaderboards for sound demixing tasks](https://mlsanity.com/paper/2305.07489/) (2023)
- [GASS: Generalizing Audio Source Separation with Large-scale Data](https://mlsanity.com/paper/2310.00140/) (2023)
- [Listen, Chat, and Remix: Text-Guided Soundscape Remixing for Enhanced Auditory Experience](https://mlsanity.com/paper/2402.03710/) (2024)
- [Learning Source Disentanglement in Neural Audio Codec](https://mlsanity.com/paper/2409.11228/) (2024)
- [Remastering Divide and Remaster: A Cinematic Audio Source Separation Dataset with Multilingual Support](https://mlsanity.com/paper/2407.07275/) (2024)
- [Disentangling the Impacts of Language and Channel Variability on Speech Separation Networks](https://mlsanity.com/paper/2203.16040/) (2022)
- [Late Audio-Visual Fusion for In-The-Wild Speaker Diarization](https://mlsanity.com/paper/2211.01299/) (2022)
- [SPMamba: State-space model is all you need in speech separation](https://mlsanity.com/paper/2404.02063/) (2024)
- [An Ensemble Approach to Music Source Separation: A Comparative Analysis of Conventional and Hierarchical Stem Separation](https://mlsanity.com/paper/2410.20773/) (2024)
- [MoXaRt: Audio-Visual Object-Guided Sound Interaction for XR](https://mlsanity.com/paper/2603.10465/) (2026)
- [Facing the Music: Tackling Singing Voice Separation in Cinematic Audio Source Separation](https://mlsanity.com/paper/2408.03588/) (2024)
- [ConcateNet: Dialogue Separation Using Local And Global Feature Concatenation](https://mlsanity.com/paper/2408.08729/) (2024)
- [A Knowledge-Driven Approach to Music Segmentation, Music Source Separation and Cinematic Audio Source Separation](https://mlsanity.com/paper/2602.21476/) (2026)

## Abstract (arXiv)

The cocktail party problem aims at isolating any source of interest within a complex acoustic scene, and has long inspired audio source separation research. Recent efforts have mainly focused on separating speech from noise, speech from speech, musical instruments from each other, or sound events from each other. However, separating an audio mixture (e.g., movie soundtrack) into the three broad categories of speech, music, and sound effects (understood to include ambient noise and natural sound events) has been left largely unexplored, despite a wide range of potential applications. This paper formalizes this task as the cocktail fork problem, and presents the Divide and Remaster (DnR) dataset to foster research on this topic. DnR is built from three well-established audio datasets (LibriSpeech, FMA, FSD50k), taking care to reproduce conditions similar to professionally produced content in terms of source overlap and relative loudness, and made available at CD quality. We benchmark standard source separation algorithms on DnR, and further introduce a new multi-resolution model to better address the variety of acoustic characteristics of the three source types. Our best model produces SI-SDR improvements over the mixture of 11.0 dB for music, 11.2 dB for speech, and 10.8 dB for sound effects.