# FasTUSS: Faster Task-Aware Unified Source Separation

arXiv: https://arxiv.org/abs/2507.11435 · HTML: https://mlsanity.com/paper/2507.11435/

## Key facts

- Cited by 2 papers in the map (data through 19 Oct 2026).
- Placed in Vocoders and speech synthesis, one of 11 directions in Speech and audio language models.
- Builds on 48 of its 54 references that are in the map.
- First posted on arXiv 15 Jul 2025.

## Placed in

- Speech and audio language models / [Vocoders and speech synthesis](https://mlsanity.com/speech/vocoders-and-speech-synthesis/)

## Builds on (most cited first, 30 of 48)

- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Xception: Deep Learning with Depthwise Separable Convolutions](https://mlsanity.com/paper/1610.02357/) (2016)
- [ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devices](https://mlsanity.com/paper/1707.01083/) (2017)
- [ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design](https://mlsanity.com/paper/1807.11164/) (2018)
- [FSD50K: An Open Dataset of Human-Labeled Sound Events](https://mlsanity.com/paper/2010.00475/) (2020)
- [Supervised Speech Separation Based on Deep Learning: An Overview](https://mlsanity.com/paper/1708.07524/) (2017)
- [Deep clustering: Discriminative embeddings for segmentation and separation](https://mlsanity.com/paper/1508.04306/) (2015)
- [Dual-path RNN: efficient long sequence modeling for time-domain single-channel speech separation](https://mlsanity.com/paper/1910.06379/) (2019)
- [MCUNet: Tiny Deep Learning on IoT Devices](https://mlsanity.com/paper/2007.10319/) (2020)
- [Permutation Invariant Training of Deep Models for Speaker-Independent Multi-talker Speech Separation](https://mlsanity.com/paper/1607.00325/) (2016)
- [WHAM!: Extending Speech Separation to Noisy Environments](https://mlsanity.com/paper/1907.01160/) (2019)
- [Attention is All You Need in Speech Separation](https://mlsanity.com/paper/2010.13154/) (2020)
- [FMA: A Dataset For Music Analysis](https://mlsanity.com/paper/1612.01840/) (2016)
- [The INTERSPEECH 2020 Deep Noise Suppression Challenge: Datasets, Subjective Testing Framework, and Challenge Results](https://mlsanity.com/paper/2005.13981/) (2020)
- [EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attention](https://mlsanity.com/paper/2305.07027/) (2023)
- [VoiceFilter: Targeted Voice Separation by Speaker-Conditioned Spectrogram Masking](https://mlsanity.com/paper/1810.04826/) (2018)
- [TF-GridNet: Integrating Full- and Sub-Band Modeling for Speech Separation](https://mlsanity.com/paper/2211.12433/) (2022)
- [Luna: Linear Unified Nested Attention](https://mlsanity.com/paper/2106.01540/) (2021)
- [Music Source Separation with Band-split RNN](https://mlsanity.com/paper/2209.15174/) (2022)
- [Sudo rm -rf: Efficient Networks for Universal Audio Source Separation](https://mlsanity.com/paper/2007.06833/) (2020)
- [Universal Sound Separation](https://mlsanity.com/paper/1905.03330/) (2019)
- [Separate What You Describe: Language-Queried Audio Source Separation](https://mlsanity.com/paper/2203.15147/) (2022)
- [Moisesdb: A dataset for source separation beyond 4-stems](https://mlsanity.com/paper/2307.15913/) (2023)
- [URGENT Challenge: Universality, Robustness, and Generalizability For Speech Enhancement](https://mlsanity.com/paper/2406.04660/) (2024)
- [Music Source Separation with Band-Split RoPE Transformer](https://mlsanity.com/paper/2309.02612/) (2023)
- [Listen to What You Want: Neural Network-based Universal Sound Selector](https://mlsanity.com/paper/2006.05712/) (2020)
- [Toward Universal Speech Enhancement for Diverse Input Conditions](https://mlsanity.com/paper/2309.17384/) (2023)
- [Zero-shot Audio Source Separation through Query-based Learning from Weakly-labeled Data](https://mlsanity.com/paper/2112.07891/) (2021)
- [MicroNet: Improving Image Recognition with Extremely Low FLOPs](https://mlsanity.com/paper/2108.05894/) (2021)

## Built on by (most cited first, 2 of 2)

- [SUNAC: Source-aware Unified Neural Audio Codec](https://mlsanity.com/paper/2511.16126/) (2025)
- [Downstream-Task-Aware Unified Source Separation](https://mlsanity.com/paper/2609.11092/) (2026)

## Abstract (arXiv)

Time-Frequency (TF) dual-path models are currently among the best performing audio source separation network architectures, achieving state-of-the-art performance in speech enhancement, music source separation, and cinematic audio source separation. While they are characterized by a relatively low parameter count, they still require a considerable number of operations, implying a higher execution time. This problem is exacerbated by the trend towards bigger models trained on large amounts of data to solve more general tasks, such as the recently introduced task-aware unified source separation (TUSS) model. TUSS, which aims to solve audio source separation tasks using a single, conditional model, is built upon TF-Locoformer, a TF dual-path model combining convolution and attention layers. The task definition comes in the form of a sequence of prompts that specify the number and type of sources to be extracted. In this paper, we analyze the design choices of TUSS with the goal of optimizing its performance-complexity trade-off. We derive two more efficient models, FasTUSS-8.3G and FasTUSS-11.7G that reduce the original model's operations by 81% and 73% with minor performance drops of 1.2~dB and 0.4~dB averaged over all benchmarks, respectively. Additionally, we investigate the impact of prompt conditioning to derive a causal TUSS model.