# Listen to What You Want: Neural Network-based Universal Sound Selector

arXiv: https://arxiv.org/abs/2006.05712 · HTML: https://mlsanity.com/paper/2006.05712/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Jun 2020.

## Built on by (most cited first, 27 of 27)

- [Separate What You Describe: Language-Queried Audio Source Separation](https://mlsanity.com/paper/2203.15147/) (2022)
- [Separate Anything You Describe](https://mlsanity.com/paper/2308.05037/) (2023)
- [Move2Hear: Active Audio-Visual Source Separation](https://mlsanity.com/paper/2105.07142/) (2021)
- [AudioScopeV2: Audio-Visual Attention Architectures for Calibrated Open-Domain On-Screen Sound Separation](https://mlsanity.com/paper/2207.10141/) (2022)
- [CLIPSep: Learning Text-queried Sound Separation with Noisy Unlabeled Videos](https://mlsanity.com/paper/2212.07065/) (2022)
- [Real-Time Target Sound Extraction](https://mlsanity.com/paper/2211.02250/) (2022)
- [FlowSep: Language-Queried Sound Separation with Rectified Flow Matching](https://mlsanity.com/paper/2409.07614/) (2024)
- [Active Audio-Visual Separation of Dynamic Sound Sources](https://mlsanity.com/paper/2202.00850/) (2022)
- [Heterogeneous Target Speech Separation](https://mlsanity.com/paper/2204.03594/) (2022)
- [DPM-TSE: A Diffusion Probabilistic Model for Target Sound Extraction](https://mlsanity.com/paper/2310.04567/) (2023)
- [Semantic Hearing: Programming Acoustic Scenes with Binaural Hearables](https://mlsanity.com/paper/2311.00320/) (2023)
- [SoloAudio: Target Sound Extraction with Language-oriented Audio Diffusion Transformer](https://mlsanity.com/paper/2409.08425/) (2024)
- [OmniSep: Unified Omni-Modality Sound Separation with Query-Mixup](https://mlsanity.com/paper/2410.21269/) (2024)
- [Typing to Listen at the Cocktail Party: Text-Guided Target Speaker Extraction](https://mlsanity.com/paper/2310.07284/) (2023)
- [Improving On-Screen Sound Separation for Open-Domain Videos with Audio-Visual Self-Attention](https://mlsanity.com/paper/2106.09669/) (2021)
- [Optimal Condition Training for Target Source Separation](https://mlsanity.com/paper/2211.05927/) (2022)
- [MGE-LDM: Joint Latent Diffusion for Simultaneous Music Generation and Source Extraction](https://mlsanity.com/paper/2505.23305/) (2025)
- [Environmental Sound Extraction Using Onomatopoeic Words](https://mlsanity.com/paper/2112.00209/) (2021)
- [Multichannel-to-Multichannel Target Sound Extraction Using Direction and Timestamp Clues](https://mlsanity.com/paper/2409.12415/) (2024)
- [Locate This, Not That: Class-Conditioned Sound Event DOA Estimation](https://mlsanity.com/paper/2203.04197/) (2022)
- [Unleashing the Power of Natural Audio Featuring Multiple Sound Sources](https://mlsanity.com/paper/2504.17782/) (2025)
- [FlowSep 2: Self-Supervised Flow Matching for Language-Queried Audio Source Separation](https://mlsanity.com/paper/2608.22111/) (2026)
- [SANE-TTS: Stable And Natural End-to-End Multilingual Text-to-Speech](https://mlsanity.com/paper/2206.12132/) (2022)
- [Complete and separate: Conditional separation with missing target source attribute completion](https://mlsanity.com/paper/2307.14609/) (2023)
- [CATSE: A Context-Aware Framework for Causal Target Sound Extraction](https://mlsanity.com/paper/2403.14246/) (2024)
- [SoundCompass: Navigating Target Sound Extraction With Effective Directional Clue Integration In Complex Acoustic Scenes](https://mlsanity.com/paper/2509.18561/) (2025)
- [Fine-grained Soundscape Control for Augmented Hearing](https://mlsanity.com/paper/2603.00395/) (2026)

## Abstract (arXiv)

Being able to control the acoustic events (AEs) to which we want to listen would allow the development of more controllable hearable devices. This paper addresses the AE sound selection (or removal) problems, that we define as the extraction (or suppression) of all the sounds that belong to one or multiple desired AE classes. Although this problem could be addressed with a combination of source separation followed by AE classification, this is a sub-optimal way of solving the problem. Moreover, source separation usually requires knowing the maximum number of sources, which may not be practical when dealing with AEs. In this paper, we propose instead a universal sound selection neural network that enables to directly select AE sounds from a mixture given user-specified target AE classes. The proposed framework can be explicitly optimized to simultaneously select sounds from multiple desired AE classes, independently of the number of sources in the mixture. We experimentally show that the proposed method achieves promising AE sound selection performance and could be generalized to mixtures with a number of sources that are unseen during training.