# Downstream-Task-Aware Unified Source Separation

arXiv: https://arxiv.org/abs/2609.11092 · HTML: https://mlsanity.com/paper/2609.11092/

## Key facts

- No citing papers in the map yet (data through 20 Oct 2026).
- Placed in Speech enhancement and noise suppression, one of 11 directions in Speech and audio language models.
- Builds on 18 of its 24 references that are in the map.
- First posted on arXiv 10 Sep 2026.

## Placed in

- Speech and audio language models / [Speech enhancement and noise suppression](https://mlsanity.com/speech/speech-enhancement-and-noise-suppression/)

## Builds on (most cited first, 18 of 18)

- [SDR - half-baked or well done?](https://mlsanity.com/paper/1811.02508/) (2018)
- [Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation](https://mlsanity.com/paper/1809.07454/) (2018)
- [UTMOS: UTokyo-SaruLab System for VoiceMOS Challenge 2022](https://mlsanity.com/paper/2204.02152/) (2022)
- [Supervised Speech Separation Based on Deep Learning: An Overview](https://mlsanity.com/paper/1708.07524/) (2017)
- [Deep clustering: Discriminative embeddings for segmentation and separation](https://mlsanity.com/paper/1508.04306/) (2015)
- [Dual-path RNN: efficient long sequence modeling for time-domain single-channel speech separation](https://mlsanity.com/paper/1910.06379/) (2019)
- [Permutation Invariant Training of Deep Models for Speaker-Independent Multi-talker Speech Separation](https://mlsanity.com/paper/1607.00325/) (2016)
- [Attention is All You Need in Speech Separation](https://mlsanity.com/paper/2010.13154/) (2020)
- [Music Source Separation with Band-split RNN](https://mlsanity.com/paper/2209.15174/) (2022)
- [How Bad Are Artifacts?: Analyzing the Impact of Speech Enhancement Errors on ASR](https://mlsanity.com/paper/2201.06685/) (2022)
- [Toward Universal Speech Enhancement for Diverse Input Conditions](https://mlsanity.com/paper/2309.17384/) (2023)
- [TF-Locoformer: Transformer with Local Modeling by Convolution for Speech Separation and Enhancement](https://mlsanity.com/paper/2408.03440/) (2024)
- [TIGER: Time-frequency Interleaved Gain Extraction and Reconstruction for Efficient Speech Separation](https://mlsanity.com/paper/2410.01469/) (2024)
- [Rethinking Processing Distortions: Disentangling the Impact of Speech Enhancement Errors on Speech Recognition Performance](https://mlsanity.com/paper/2404.14860/) (2024)
- [SNRi Target Training for Joint Speech Enhancement and Recognition](https://mlsanity.com/paper/2111.00764/) (2021)
- [Reducing the Gap Between Pretrained Speech Enhancement and Recognition Models Using a Real Speech-Trained Bridging Module](https://mlsanity.com/paper/2501.02452/) (2025)
- [Task-Aware Unified Source Separation](https://mlsanity.com/paper/2410.23987/) (2024)
- [FasTUSS: Faster Task-Aware Unified Source Separation](https://mlsanity.com/paper/2507.11435/) (2025)

## Abstract (arXiv)

Task-aware unified source separation (TUSS) enables a single model to handle diverse separation tasks by conditioning on input prompts. However, conventional TUSS does not account for downstream task requirements, such as whether the enhanced speech will be used for human listening or automatic speech recognition (ASR). In this paper, we propose a prompt extension framework for TUSS that incorporates downstream task information into the input prompts and switches the loss function according to the given prompt during training, enabling outputs with different signal characteristics at inference time. Specifically, we introduce an ASR-dedicated prompt paired with a regularized loss function that reduces speech artifacts to improve ASR robustness, while the standard prompt is paired with the conventional SNR loss function. Experiments on the LibriSpeech and JNAS corpora demonstrate that the proposed joint-training scheme enables a single model to improve ASR performance over noisy input across a wide range of SNR conditions by selecting the ASR-dedicated prompt, while maintaining general speech enhancement quality when the standard prompt is used.