# An empirical study of Conv-TasNet

arXiv: https://arxiv.org/abs/2002.08688 · HTML: https://mlsanity.com/paper/2002.08688/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Feb 2020.

## Built on by (most cited first, 15 of 15)

- [On loss functions and evaluation metrics for music source separation](https://mlsanity.com/paper/2202.07968/) (2022)
- [TIGER: Time-frequency Interleaved Gain Extraction and Reconstruction for Efficient Speech Separation](https://mlsanity.com/paper/2410.01469/) (2024)
- [Adversarial Permutation Invariant Training for Universal Sound Separation](https://mlsanity.com/paper/2210.12108/) (2022)
- [GASS: Generalizing Audio Source Separation with Large-scale Data](https://mlsanity.com/paper/2310.00140/) (2023)
- [TrustEMG-Net: Using Representation-Masking Transformer with U-Net for Surface Electromyography Enhancement](https://mlsanity.com/paper/2410.03843/) (2024)
- [Directed Speech Separation for Automatic Speech Recognition of Long Form Conversational Speech](https://mlsanity.com/paper/2112.05863/) (2021)
- [On permutation invariant training for speech source separation](https://mlsanity.com/paper/2102.04945/) (2021)
- [Quantitative Evidence on Overlooked Aspects of Enrollment Speaker Embeddings for Target Speaker Separation](https://mlsanity.com/paper/2210.12635/) (2022)
- [LibriheavyMix: A 20,000-Hour Dataset for Single-Channel Reverberant Multi-Talker Speech Separation, ASR and Speaker Diarization](https://mlsanity.com/paper/2409.00819/) (2024)
- [A Study of the Scale Invariant Signal to Distortion Ratio in Speech Separation with Noisy References](https://mlsanity.com/paper/2508.14623/) (2025)
- [Disentangling the Impacts of Language and Channel Variability on Speech Separation Networks](https://mlsanity.com/paper/2203.16040/) (2022)
- [MB-RIRs: a Synthetic Room Impulse Response Dataset with Frequency-Dependent Absorption Coefficients](https://mlsanity.com/paper/2507.09750/) (2025)
- [CasNet: Investigating Channel Robustness for Speech Separation](https://mlsanity.com/paper/2210.15370/) (2022)
- [Conv-NILM-Net, a causal and multi-appliance model for energy source separation](https://mlsanity.com/paper/2208.02173/) (2022)
- [DISPATCH: Distilling Selective Patches for Speech Enhancement](https://mlsanity.com/paper/2509.15922/) (2025)

## Abstract (arXiv)

Conv-TasNet is a recently proposed waveform-based deep neural network that achieves state-of-the-art performance in speech source separation. Its architecture consists of a learnable encoder/decoder and a separator that operates on top of this learned space. Various improvements have been proposed to Conv-TasNet. However, they mostly focus on the separator, leaving its encoder/decoder as a (shallow) linear operator. In this paper, we conduct an empirical study of Conv-TasNet and propose an enhancement to the encoder/decoder that is based on a (deep) non-linear variant of it. In addition, we experiment with the larger and more diverse LibriTTS dataset and investigate the generalization capabilities of the studied models when trained on a much larger dataset. We propose cross-dataset evaluation that includes assessing separations from the WSJ0-2mix, LibriTTS and VCTK databases. Our results show that enhancements to the encoder/decoder can improve average SI-SNR performance by more than 1 dB. Furthermore, we offer insights into the generalization capabilities of Conv-TasNet and the potential value of improvements to the encoder/decoder.