# One-shot conditional audio filtering of arbitrary sounds

arXiv: https://arxiv.org/abs/2011.02421 · HTML: https://mlsanity.com/paper/2011.02421/

## Key facts

- Cited by 18 papers in the map (data through 13 Oct 2026).
- First posted on arXiv 4 Nov 2020.

## Built on by (most cited first, 18 of 18)

- [Separate Anything You Describe](https://mlsanity.com/paper/2308.05037/) (2023)
- [Transformers in Speech Processing: A Survey](https://mlsanity.com/paper/2303.11607/) (2023)
- [AudioScopeV2: Audio-Visual Attention Architectures for Calibrated Open-Domain On-Screen Sound Separation](https://mlsanity.com/paper/2207.10141/) (2022)
- [CLIPSep: Learning Text-queried Sound Separation with Noisy Unlabeled Videos](https://mlsanity.com/paper/2212.07065/) (2022)
- [Real-Time Target Sound Extraction](https://mlsanity.com/paper/2211.02250/) (2022)
- [DPM-TSE: A Diffusion Probabilistic Model for Target Sound Extraction](https://mlsanity.com/paper/2310.04567/) (2023)
- [Semantic Hearing: Programming Acoustic Scenes with Binaural Hearables](https://mlsanity.com/paper/2311.00320/) (2023)
- [SoloAudio: Target Sound Extraction with Language-oriented Audio Diffusion Transformer](https://mlsanity.com/paper/2409.08425/) (2024)
- [Multistage linguistic conditioning of convolutional layers for speech emotion recognition](https://mlsanity.com/paper/2110.06650/) (2021)
- [A Stem-Agnostic Single-Decoder System for Music Source Separation Beyond Four Stems](https://mlsanity.com/paper/2406.18747/) (2024)
- [Weakly-supervised Audio Separation via Bi-modal Semantic Similarity](https://mlsanity.com/paper/2404.01740/) (2024)
- [Cross-attention Inspired Selective State Space Models for Target Sound Extraction](https://mlsanity.com/paper/2409.04803/) (2024)
- [Do You See What I See? Capabilities and Limits of Automated Multimedia Content Analysis](https://mlsanity.com/paper/2201.11105/) (2021)
- [OpenSep: Leveraging Large Language Models with Textual Inversion for Open World Audio Separation](https://mlsanity.com/paper/2409.19270/) (2024)
- [FlowSep 2: Self-Supervised Flow Matching for Language-Queried Audio Source Separation](https://mlsanity.com/paper/2608.22111/) (2026)
- [CATSE: A Context-Aware Framework for Causal Target Sound Extraction](https://mlsanity.com/paper/2403.14246/) (2024)
- [Text-Queried Audio Source Separation via Hierarchical Modeling](https://mlsanity.com/paper/2505.21025/) (2025)
- [Fine-grained Soundscape Control for Augmented Hearing](https://mlsanity.com/paper/2603.00395/) (2026)

## Abstract (arXiv)

We consider the problem of separating a particular sound source from a single-channel mixture, based on only a short sample of the target source. Using SoundFilter, a wave-to-wave neural network architecture, we can train a model without using any sound class labels. Using a conditioning encoder model which is learned jointly with the source separation network, the trained model can be "configured" to filter arbitrary sound sources, even ones that it has not seen during training. Evaluated on the FSD50k dataset, our model obtains an SI-SDR improvement of 9.6 dB for mixtures of two sounds. When trained on Librispeech, our model achieves an SI-SDR improvement of 14.0 dB when separating one voice from a mixture of two speakers. Moreover, we show that the representation learned by the conditioning encoder clusters acoustically similar sounds together in the embedding space, even though it is trained without using any labels.