mlsanityData to 7 Oct 2026

Resource-Efficient Separation Transformer

arXiv 2206.09507 ↗First posted 19 Jun 2022eess.AS cs.LG cs.SDcited by 11

Not on the map. Why?

Abstract (arXiv)

Transformers have recently achieved state-of-the-art performance in speech separation. These models, however, are computationally demanding and require a lot of learnable parameters. This paper explores Transformer-based speech separation with a reduced computational cost. Our main contribution is the development of the Resource-Efficient Separation Transformer (RE-SepFormer), a self-attention-based architecture that reduces the computational burden in two ways. First, it uses non-overlapping blocks in the latent space. Second, it operates on compact latent summaries calculated from each chunk. The RE-SepFormer reaches a competitive performance on the popular WSJ0-2Mix and WHAM! datasets in both causal and non-causal settings. Remarkably, it scales significantly better than the previous Transformer-based architectures in terms of memory and inference time, making it more suitable for processing long mixtures.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 41 of its 56 references that are in the map. First posted on arXiv 19 Jun 2022.

Builds on

41 of 56 references.

References · 41

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2020Language Models are Few-Shot Learnerscited by 32,199
  4. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  6. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
  7. 2020Longformer: The Long-Document Transformercited by 3,362
  8. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,301
  9. 2019Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelismcited by 1,829
  10. 2020Reformer: The Efficient Transformercited by 1,658
  11. 2020Linformer: Self-Attention with Linear Complexitycited by 1,462
  12. 2020Efficient Transformers: A Surveycited by 784
29 more
  1. 2018SDR - half-baked or well done?cited by 420
  2. 2016Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activationscited by 413
  3. 2021SpeechBrain: A General-Purpose Speech Toolkitcited by 404
  4. 2018Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separationcited by 400
  5. 2015Deep clustering: Discriminative embeddings for segmentation and separationcited by 211
  6. 2020Attention is All You Need in Speech Separationcited by 184
  7. 2019Dual-path RNN: efficient long sequence modeling for time-domain single-channel speech separationcited by 182
  8. 2019WHAM!: Extending Speech Separation to Noisy Environmentscited by 154
  9. 2020DCCRN: Deep Complex Convolution Recurrent Network for Phase-Aware Speech Enhancementcited by 133
  10. 2017TasNet: time-domain audio separation network for real-time, single-channel speech separationcited by 115
  11. 2017Multi-talker Speech Separation with Utterance-level Permutation Invariant Training of Deep Recurrent Neural Networkscited by 97
  12. 2020ESPnet-ST: All-in-One Speech Translation Toolkitcited by 82
  13. 2020Dual-Path Transformer Network: Direct Context-Aware Modeling for End-to-End Monaural Speech Separationcited by 67
  14. 2020Sudo rm -rf: Efficient Networks for Universal Audio Source Separationcited by 50
  15. 2022Squeezeformer: An Efficient Transformer for Automatic Speech Recognitioncited by 50
  16. 2019PHASEN: A Phase-and-Harmonics-Aware Speech Enhancement Networkcited by 49
  17. 2020Introducing the VoicePrivacy Initiativecited by 48
  18. 2016Bitwise Neural Networkscited by 29
  19. 2019Divide and Conquer: A Deep CASA Approach to Talker-independent Monaural Speaker Separationcited by 20
  20. 2018Convolutional-Recurrent Neural Networks for Speech Enhancementcited by 17
  21. 2019Two-Step Sound Source Separation: Training on Learned Latent Targetscited by 17
  22. 2022SkiM: Skipping Memory LSTM for Low-Latency Real-Time Continuous Speech Separationcited by 16
  23. 2022SepIt: Approaching a Single Channel Speech Separation Boundcited by 16
  24. 2019FurcaNeXt: End-to-end monaural speech separation with dynamic gated dilated temporal convolutional networkscited by 15
  25. 2018Deep Learning Based Phase Reconstruction for Speaker Separation: A Trigonometric Perspectivecited by 13
  26. 2022Neural Architecture Search For LF-MMI Trained Time Delay Neural Networkscited by 11
  27. 2020Ultra-Lightweight Speech Separation via Group Communicationcited by 8
  28. 2022Neural Architecture Search for Speech Emotion Recognitioncited by 6
  29. 2022Exploring Self-Attention Mechanisms for Speech Separationcited by 4

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 1; 2024: 3; 2025: 4.

11 papers.

  1. 2023Transformers in Speech Processing: A Surveycited by 38
  2. 2022Real-Time Target Sound Extractioncited by 22
  3. 2024Neural Blind Source Separation and Diarization for Distant Speech Recognitioncited by 6
  4. 2024Target conversation extraction: Source separation using turn-taking dynamicscited by 4
  5. 2022Deformable Temporal Convolutional Networks for Monaural Noisy Reverberant Speech Separationcited by 3
  6. 2022Deep neural network techniques for monaural speech enhancement: state of the art analysiscited by 3
  7. 2025Neural Speech Extraction with Human Feedbackcited by 2
  8. 2025HiFi-Stream: Streaming Speech Enhancement with Generative Adversarial Networkscited by 1
  9. 2025Time-Frequency-Based Attention Cache Memory Model for Real-Time Speech Separationcited by 1
  10. 2024Effects of Dataset Sampling Rate for Noise Cancellation through Deep Learningcited by 0
  11. 2025Causal Self-supervised Pretrained Frontend with Predictive Code for Speech Separationcited by 0