mlsanityData to 19 Oct 2026

FasTUSS: Faster Task-Aware Unified Source Separation

arXiv 2507.11435 ↗First posted 15 Jul 2025cs.SD eess.AS eess.SPcited by 2

On the map: Speech and audio language models → Vocoders and speech synthesis

Abstract (arXiv)

Time-Frequency (TF) dual-path models are currently among the best performing audio source separation network architectures, achieving state-of-the-art performance in speech enhancement, music source separation, and cinematic audio source separation. While they are characterized by a relatively low parameter count, they still require a considerable number of operations, implying a higher execution time. This problem is exacerbated by the trend towards bigger models trained on large amounts of data to solve more general tasks, such as the recently introduced task-aware unified source separation (TUSS) model. TUSS, which aims to solve audio source separation tasks using a single, conditional model, is built upon TF-Locoformer, a TF dual-path model combining convolution and attention layers. The task definition comes in the form of a sequence of prompts that specify the number and type of sources to be extracted. In this paper, we analyze the design choices of TUSS with the goal of optimizing its performance-complexity trade-off. We derive two more efficient models, FasTUSS-8.3G and FasTUSS-11.7G that reduce the original model's operations by 81% and 73% with minor performance drops of 1.2~dB and 0.4~dB averaged over all benchmarks, respectively. Additionally, we investigate the impact of prompt conditioning to derive a causal TUSS model.

Key facts. Cited by 2 papers in the map (data through 19 Oct 2026). Placed in Vocoders and speech synthesis, one of 11 directions in Speech and audio language models. Builds on 48 of its 54 references that are in the map. First posted on arXiv 15 Jul 2025.

Builds on

48 of 54 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024Task-Aware Unified Source Separation31 Oct
  2. 2021Multi-Task Audio Source Separation14 Jul

Other references · 46

  1. 2023Efficient Memory Management for Large Language Model Serving with PagedAttentioncited by 6,066
  2. 2021RoFormer: Enhanced Transformer with Rotary Position Embeddingcited by 4,812
  3. 2016Xception: Deep Learning with Depthwise Separable Convolutionscited by 2,349
  4. 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,204
  5. 2018ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Designcited by 1,143
  6. 2020FSD50K: An Open Dataset of Human-Labeled Sound Eventscited by 395
  7. 2017Supervised Speech Separation Based on Deep Learning: An Overviewcited by 342
  8. 2015Deep clustering: Discriminative embeddings for segmentation and separationcited by 281
  9. 2019Dual-path RNN: efficient long sequence modeling for time-domain single-channel speech separationcited by 275
  10. 2020MCUNet: Tiny Deep Learning on IoT Devicescited by 275
  11. 2016Permutation Invariant Training of Deep Models for Speaker-Independent Multi-talker Speech Separationcited by 247
  12. 2019WHAM!: Extending Speech Separation to Noisy Environmentscited by 246
34 more
  1. 2020Attention is All You Need in Speech Separationcited by 236
  2. 2016FMA: A Dataset For Music Analysiscited by 234
  3. 2020The INTERSPEECH 2020 Deep Noise Suppression Challenge: Datasets, Subjective Testing Framework, and Challenge Resultscited by 230
  4. 2023EfficientViT: Memory Efficient Vision Transformer with Cascaded Group Attentioncited by 174
  5. 2018VoiceFilter: Targeted Voice Separation by Speaker-Conditioned Spectrogram Maskingcited by 122
  6. 2022TF-GridNet: Integrating Full- and Sub-Band Modeling for Speech Separationcited by 121
  7. 2021Luna: Linear Unified Nested Attentioncited by 98
  8. 2022Music Source Separation with Band-split RNNcited by 89
  9. 2020Sudo rm -rf: Efficient Networks for Universal Audio Source Separationcited by 64
  10. 2019Universal Sound Separationcited by 51
  11. 2022Separate What You Describe: Language-Queried Audio Source Separationcited by 50
  12. 2023Moisesdb: A dataset for source separation beyond 4-stemscited by 44
  13. 2024URGENT Challenge: Universality, Robustness, and Generalizability For Speech Enhancementcited by 44
  14. 2023Music Source Separation with Band-Split RoPE Transformercited by 43
  15. 2020Listen to What You Want: Neural Network-based Universal Sound Selectorcited by 32
  16. 2023Toward Universal Speech Enhancement for Diverse Input Conditionscited by 32
  17. 2021Zero-shot Audio Source Separation through Query-based Learning from Weakly-labeled Datacited by 29
  18. 2021MicroNet: Improving Image Recognition with Extremely Low FLOPscited by 27
  19. 2022Text-Driven Separation of Arbitrary Soundscited by 26
  20. 2020All for One and One for All: Improving Music Separation by Bridging Networkscited by 25
  21. 2021The Cocktail Fork Problem: Three-Stem Audio Separation for Real-World Soundtrackscited by 24
  22. 2019Improving Universal Sound Separation Using Sound Classificationcited by 21
  23. 2024TF-Locoformer: Transformer with Local Modeling by Convolution for Speech Separation and Enhancementcited by 20
  24. 2019Audio query-based music source separationcited by 18
  25. 2021PhiNets: a scalable backbone for low-power AI at the edgecited by 17
  26. 2022Heterogeneous Target Speech Separationcited by 17
  27. 2022SoundBeam: Target sound extraction conditioned on sound-class labels and enrollment clues for increased performance and continuous learningcited by 16
  28. 2018Class-conditional embeddings for music source separationcited by 14
  29. 2023The Sound Demixing Challenge 2023 unicodex2013 Cinematic Demixing Trackcited by 13
  30. 2022Few-Shot Musical Source Separationcited by 11
  31. 2023GASS: Generalizing Audio Source Separation with Large-scale Datacited by 11
  32. 2024CLAPSep: Leveraging Contrastive Pre-trained Model for Multi-Modal Query-Conditioned Target Sound Extractioncited by 11
  33. 2022Exploring Self-Attention Mechanisms for Speech Separationcited by 8
  34. 2025A Comparative Study on Positional Encoding for Time-frequency Domain Dual-path Transformer-based Source Separation Modelscited by 1

Built on by

Cited by year (2026 to 19 Oct)

Citing papers per year: 2025: 1; 2026: 1.

2 papers.

  1. 2025SUNAC: Source-aware Unified Neural Audio Codeccited by 0
  2. 2026Downstream-Task-Aware Unified Source Separationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025TF-MLPNet: Tiny Real-Time Neural Speech Separation
  2. 2025TISDiSS: A Training-Time and Inference-Time Scalable Framework for Discriminative Source Separation
  3. 2025SAM Audio: Segment Anything in Audio
  4. 2023RTFS-Net: Recurrent Time-Frequency Modelling for Efficient Audio-Visual Speech Separation
  5. 2025DGMO: Training-Free Audio Source Separation through Diffusion-Guided Mask Optimization
  6. 2024TDFNet: An Efficient Audio-Visual Speech Separation Model with Top-down Fusion
  7. 2022Tackling the Cocktail Fork Problem for Separation and Transcription of Real-World Soundtracks
  8. 2025Training-Free Multi-Step Audio Source Separation