# Tiny-Sepformer: A Tiny Time-Domain Transformer Network for Speech Separation

arXiv: https://arxiv.org/abs/2206.13689 · HTML: https://mlsanity.com/paper/2206.13689/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Jun 2022.

## Built on by (most cited first, 8 of 8)

- [Transformers in Speech Processing: A Survey](https://mlsanity.com/paper/2303.11607/) (2023)
- [A Machine Learning-oriented Survey on Tiny Machine Learning](https://mlsanity.com/paper/2309.11932/) (2023)
- [Real-Time Target Sound Extraction](https://mlsanity.com/paper/2211.02250/) (2022)
- [Semantic Hearing: Programming Acoustic Scenes with Binaural Hearables](https://mlsanity.com/paper/2311.00320/) (2023)
- [Deformable Temporal Convolutional Networks for Monaural Noisy Reverberant Speech Separation](https://mlsanity.com/paper/2210.15305/) (2022)
- [Deep neural network techniques for monaural speech enhancement: state of the art analysis](https://mlsanity.com/paper/2212.00369/) (2022)
- [Papez: Resource-Efficient Speech Separation with Auditory Working Memory](https://mlsanity.com/paper/2407.00888/) (2024)
- [TISDiSS: A Training-Time and Inference-Time Scalable Framework for Discriminative Source Separation](https://mlsanity.com/paper/2509.15666/) (2025)

## Abstract (arXiv)

Time-domain Transformer neural networks have proven their superiority in speech separation tasks. However, these models usually have a large number of network parameters, thus often encountering the problem of GPU memory explosion. In this paper, we proposed Tiny-Sepformer, a tiny version of Transformer network for speech separation. We present two techniques to reduce the model parameters and memory consumption: (1) Convolution-Attention (CA) block, spliting the vanilla Transformer to two paths, multi-head attention and 1D depthwise separable convolution, (2) parameter sharing, sharing the layer parameters within the CA block. In our experiments, Tiny-Sepformer could greatly reduce the model size, and achieves comparable separation performance with vanilla Sepformer on WSJ0-2/3Mix datasets.