# TasNet: time-domain audio separation network for real-time, single-channel speech separation

arXiv: https://arxiv.org/abs/1711.00541 · HTML: https://mlsanity.com/paper/1711.00541/

## Key facts

- Cited by 115 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Nov 2017.

## Built on by (most cited first, 30 of 115)

- [Dual-path Mamba: Short and Long-term Bidirectional Selective Structured State Space Models for Speech Separation](https://mlsanity.com/paper/2403.18257/) (2024)
- [MambaMixer: Efficient Selective State Space Models with Dual Token and Channel Selection](https://mlsanity.com/paper/2403.19888/) (2024)
- [An efficient encoder-decoder architecture with top-down attention for speech separation](https://mlsanity.com/paper/2209.15200/) (2022)
- [Looking into Your Speech: Learning Cross-modal Affinity for Audio-visual Speech Separation](https://mlsanity.com/paper/2104.02775/) (2021)
- [Zero-shot Audio Source Separation through Query-based Learning from Weakly-labeled Data](https://mlsanity.com/paper/2112.07891/) (2021)
- [MossFormer: Pushing the Performance Limit of Monaural Speech Separation using Gated Single-Head Transformer with Convolution-Augmented Joint Self-Attentions](https://mlsanity.com/paper/2302.11824/) (2023)
- [Deep Impulse Responses: Estimating and Parameterizing Filters with Deep Networks](https://mlsanity.com/paper/2202.03416/) (2022)
- [End-to-End Integration of Speech Recognition, Speech Enhancement, and Self-Supervised Learning Representation](https://mlsanity.com/paper/2204.00540/) (2022)
- [SepIt: Approaching a Single Channel Speech Separation Bound](https://mlsanity.com/paper/2205.11801/) (2022)
- [Towards Neural Diarization for Unlimited Numbers of Speakers Using Global and Local Attractors](https://mlsanity.com/paper/2107.01545/) (2021)
- [Separate And Diffuse: Using a Pretrained Diffusion Model for Improving Source Separation](https://mlsanity.com/paper/2301.10752/) (2023)
- [A Sidecar Separator Can Convert a Single-Talker Speech Recognition System to a Multi-Talker One](https://mlsanity.com/paper/2302.09908/) (2023)
- [PEAVS: Perceptual Evaluation of Audio-Visual Synchrony Grounded in Viewers' Opinion Scores](https://mlsanity.com/paper/2404.07336/) (2024)
- [HyperSound: Generating Implicit Neural Representations of Audio Signals with Hypernetworks](https://mlsanity.com/paper/2211.01839/) (2022)
- [Many-Speakers Single Channel Speech Separation with Optimal Permutation Training](https://mlsanity.com/paper/2104.08955/) (2021)
- [MoRe-Fi: Motion-robust and Fine-grained Respiration Monitoring via Deep-Learning UWB Radar](https://mlsanity.com/paper/2111.08195/) (2021)
- [Discretization and Re-synthesis: an alternative method to solve the Cocktail Party Problem](https://mlsanity.com/paper/2112.09382/) (2021)
- [Resource-Efficient Separation Transformer](https://mlsanity.com/paper/2206.09507/) (2022)
- [Hypernetworks build Implicit Neural Representations of Sounds](https://mlsanity.com/paper/2302.04959/) (2023)
- [TRAMBA: A Hybrid Transformer and Mamba Architecture for Practical Audio and Bone Conduction Speech Super Resolution and Enhancement on Mobile and Wearable Platforms](https://mlsanity.com/paper/2405.01242/) (2024)
- [Look Once to Hear: Target Speech Hearing with Noisy Examples](https://mlsanity.com/paper/2405.06289/) (2024)
- [Multimodal Attention Fusion for Target Speaker Extraction](https://mlsanity.com/paper/2102.01326/) (2021)
- [Sandglasset: A Light Multi-Granularity Self-attentive Network For Time-Domain Speech Separation](https://mlsanity.com/paper/2103.00819/) (2021)
- [Wav2code: Restore Clean Speech Representations via Codebook Lookup for Noise-Robust ASR](https://mlsanity.com/paper/2304.04974/) (2023)
- [Wave-U-Mamba: An End-To-End Framework For High-Quality And Efficient Speech Super Resolution](https://mlsanity.com/paper/2409.09337/) (2024)
- [Effective Low-Cost Time-Domain Audio Separation Using Globally Attentive Locally Recurrent Networks](https://mlsanity.com/paper/2101.05014/) (2021)
- [Weight, Block or Unit? Exploring Sparsity Tradeoffs for Speech Enhancement on Tiny Neural Accelerators](https://mlsanity.com/paper/2111.02351/) (2021)
- [Hybrid Neural Networks for On-device Directional Hearing](https://mlsanity.com/paper/2112.05893/) (2021)
- [ESPnet-SE++: Speech Enhancement for Robust Speech Recognition, Translation, and Understanding](https://mlsanity.com/paper/2207.09514/) (2022)
- [Music Separation Enhancement with Generative Modeling](https://mlsanity.com/paper/2208.12387/) (2022)

## Abstract (arXiv)

Robust speech processing in multi-talker environments requires effective speech separation. Recent deep learning systems have made significant progress toward solving this problem, yet it remains challenging particularly in real-time, short latency applications. Most methods attempt to construct a mask for each source in time-frequency representation of the mixture signal which is not necessarily an optimal representation for speech separation. In addition, time-frequency decomposition results in inherent problems such as phase/magnitude decoupling and long time window which is required to achieve sufficient frequency resolution. We propose Time-domain Audio Separation Network (TasNet) to overcome these limitations. We directly model the signal in the time-domain using an encoder-decoder framework and perform the source separation on nonnegative encoder outputs. This method removes the frequency decomposition step and reduces the separation problem to estimation of source masks on encoder outputs which is then synthesized by the decoder. Our system outperforms the current state-of-the-art causal and noncausal speech separation algorithms, reduces the computational cost of speech separation, and significantly reduces the minimum required latency of the output. This makes TasNet suitable for applications where low-power, real-time implementation is desirable such as in hearable and telecommunication devices.