# Sampling-Frequency-Independent Universal Sound Separation

arXiv: https://arxiv.org/abs/2309.12581 · HTML: https://mlsanity.com/paper/2309.12581/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 22 of its 29 references that are in the map.
- First posted on arXiv 22 Sep 2023.

## Builds on (most cited first, 22 of 22)

- [Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation](https://mlsanity.com/paper/1809.07454/) (2018)
- [Deep clustering: Discriminative embeddings for segmentation and separation](https://mlsanity.com/paper/1508.04306/) (2015)
- [Dual-path RNN: efficient long sequence modeling for time-domain single-channel speech separation](https://mlsanity.com/paper/1910.06379/) (2019)
- [Hybrid Spectrogram and Waveform Source Separation](https://mlsanity.com/paper/2111.03600/) (2021)
- [Wavesplit: End-to-End Speech Separation by Speaker Clustering](https://mlsanity.com/paper/2002.08933/) (2020)
- [Dual-Path Transformer Network: Direct Context-Aware Modeling for End-to-End Monaural Speech Separation](https://mlsanity.com/paper/2007.13975/) (2020)
- [Universal Sound Separation](https://mlsanity.com/paper/1905.03330/) (2019)
- [Music Demixing Challenge 2021](https://mlsanity.com/paper/2108.13559/) (2021)
- [Zero-shot Audio Source Separation through Query-based Learning from Weakly-labeled Data](https://mlsanity.com/paper/2112.07891/) (2021)
- [What's All the FUSS About Free Universal Sound Separation Data?](https://mlsanity.com/paper/2011.00803/) (2020)
- [Densely connected multidilated convolutional networks for dense prediction tasks](https://mlsanity.com/paper/2011.11844/) (2020)
- [MMDenseLSTM: An efficient combination of convolutional and recurrent neural networks for audio source separation](https://mlsanity.com/paper/1805.02410/) (2018)
- [Meta-learning Extractors for Music Source Separation](https://mlsanity.com/paper/2002.07016/) (2020)
- [DF-Conformer: Integrated architecture of Conv-TasNet and Conformer using linear complexity self-attention for speech enhancement](https://mlsanity.com/paper/2106.15813/) (2021)
- [Compute and memory efficient universal sound source separation](https://mlsanity.com/paper/2103.02644/) (2021)
- [Universal Source Separation with Weakly Labelled Data](https://mlsanity.com/paper/2305.07447/) (2023)
- [Finding Strength in Weakness: Learning to Separate Sounds with Weak Supervision](https://mlsanity.com/paper/1911.02182/) (2019)
- [Channel-wise Subband Input for Better Voice and Accompaniment Separation on High Resolution Music](https://mlsanity.com/paper/2008.05216/) (2020)
- [A Multi-Phase Gammatone Filterbank for Speech Separation via TasNet](https://mlsanity.com/paper/1910.11615/) (2019)
- [JaCappella Corpus: A Japanese a Cappella Vocal Ensemble Corpus](https://mlsanity.com/paper/2211.16028/) (2022)
- [Invertible DNN-based nonlinear time-frequency transform for speech enhancement](https://mlsanity.com/paper/1911.10764/) (2019)
- [Algorithms of Sampling-Frequency-Independent Layers for Non-integer Strides](https://mlsanity.com/paper/2306.10718/) (2023)

## Abstract (arXiv)

This paper proposes a universal sound separation (USS) method capable of handling untrained sampling frequencies (SFs). The USS aims at separating arbitrary sources of different types and can be the key technique to realize a source separator that can be universally used as a preprocessor for any downstream tasks. To realize a universal source separator, there are two essential properties: universalities with respect to source types and recording conditions. The former property has been studied in the USS literature, which has greatly increased the number of source types that can be handled by a single neural network. However, the latter property (e.g., SF) has received less attention despite its necessity. Since the SF varies widely depending on the downstream tasks, the universal source separator must handle a wide variety of SFs. In this paper, to encompass the two properties, we propose an SF-independent (SFI) extension of a computationally efficient USS network, SuDoRM-RF. The proposed network uses our previously proposed SFI convolutional layers, which can handle various SFs by generating convolutional kernels in accordance with an input SF. Experiments show that signal resampling can degrade the USS performance and the proposed method works more consistently than signal-resampling-based methods for various SFs.