# PromptSep: Generative Audio Separation via Multimodal Prompting

arXiv: https://arxiv.org/abs/2511.04623 · HTML: https://mlsanity.com/paper/2511.04623/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Nov 2025.

## Built on by (most cited first, 2 of 2)

- [SAM Audio Judge: A Unified Multimodal Framework for Perceptual Evaluation of Audio Separation](https://mlsanity.com/paper/2601.19702/) (2026)
- [FlowSep 2: Self-Supervised Flow Matching for Language-Queried Audio Source Separation](https://mlsanity.com/paper/2608.22111/) (2026)

## Abstract (arXiv)

Recent breakthroughs in language-queried audio source separation (LASS) have shown that generative models can achieve higher separation audio quality than traditional masking-based approaches. However, two key limitations restrict their practical use: (1) users often require operations beyond separation, such as sound removal; and (2) relying solely on text prompts can be unintuitive for specifying sound sources. In this paper, we propose PromptSep to extend LASS into a broader framework for general-purpose sound separation. PromptSep leverages a conditional diffusion model enhanced with elaborated data simulation to enable both audio extraction and sound removal. To move beyond text-only queries, we incorporate vocal imitation as an additional and more intuitive conditioning modality for our model, by incorporating Sketch2Sound as a data augmentation strategy. Both objective and subjective evaluations on multiple benchmarks demonstrate that PromptSep achieves state-of-the-art performance in sound removal and vocal-imitation-guided source separation, while maintaining competitive results on language-queried source separation.