mlsanityData to 7 Oct 2026

Attention is All You Need in Speech Separation

arXiv 2010.13154 ↗First posted 25 Oct 2020eess.AS cs.LG cs.SDcited by 184

Not on the map. Why?

Abstract (arXiv)

Recurrent Neural Networks (RNNs) have long been the dominant architecture in sequence-to-sequence learning. RNNs, however, are inherently sequential models that do not allow parallelization of their computations. Transformers are emerging as a natural alternative to standard RNNs, replacing recurrent computations with a multi-head attention mechanism. In this paper, we propose the SepFormer, a novel RNN-free Transformer-based neural network for speech separation. The SepFormer learns short and long-term dependencies with a multi-scale approach that employs transformers. The proposed model achieves state-of-the-art (SOTA) performance on the standard WSJ0-2/3mix datasets. It reaches an SI-SNRi of 22.3 dB on WSJ0-2mix and an SI-SNRi of 19.5 dB on WSJ0-3mix. The SepFormer inherits the parallelization advantages of Transformers and achieves a competitive performance even when downsampling the encoded representation by a factor of 8. It is thus significantly faster and it is less memory-demanding than the latest speech separation systems with comparable performance.

Key facts. Cited by 184 papers in the map (data through 7 Oct 2026). First posted on arXiv 25 Oct 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 16; 2022: 25; 2023: 40; 2024: 44; 2025: 40; 2026: 19.

184 papers.

Sort:

By area:

  1. 2021SpeechBrain: A General-Purpose Speech Toolkitcited by 404
  2. 2021A Practical Survey on Faster and Lighter Transformerscited by 73
  3. 2021MT3: Multi-Task Multitrack Music Transcriptioncited by 70
  4. 2023A Comprehensive Survey on Applications of Transformers for Deep Learning Taskscited by 70
  5. 2022Music Source Separation with Band-split RNNcited by 69
  6. 2024State Space Model for New-Generation Network Alternative to Transformers: A Surveycited by 59
  7. 2023Generative Pre-training for Speech with Flow Matchingcited by 55
  8. 2023Efficient Neural Music Generationcited by 50
  9. 2024Dual-path Mamba: Short and Long-term Bidirectional Selective Structured State Space Models for Speech Separationcited by 45
  10. 2022Diffusion-based Generative Speech Source Separationcited by 41
  11. 2024MambaMixer: Efficient Selective State Space Models with Dual Token and Channel Selectioncited by 40
  12. 2022An efficient encoder-decoder architecture with top-down attention for speech separationcited by 38
38 more
  1. 2023Transformers in Speech Processing: A Surveycited by 38
  2. 2025SepPrune: Structured Pruning for Efficient Deep Speech Separationcited by 31
  3. 2023MossFormer: Pushing the Performance Limit of Monaural Speech Separation using Gated Single-Head Transformer with Convolution-Augmented Joint Self-Attentionscited by 27
  4. 2023Spiking Structured State Space Model for Monaural Speech Enhancementcited by 23
  5. 2022Real-Time Target Sound Extractioncited by 22
  6. 2024Speech Slytherin: Examining the Performance and Efficiency of Mamba for Speech Separation, Recognition, and Synthesiscited by 22
  7. 2024Building a Taiwanese Mandarin Spoken Language Model: A First Attemptcited by 22
  8. 2022RemixIT: Continual self-training of speech enhancement models via bootstrapped remixingcited by 20
  9. 2025LLaSE-G1: Incentivizing Generalization Capability for LLaMA-based Speech Enhancementcited by 19
  10. 2022Active Audio-Visual Separation of Dynamic Sound Sourcescited by 16
  11. 2022SepIt: Approaching a Single Channel Speech Separation Boundcited by 16
  12. 2021Compute and memory efficient universal sound source separationcited by 15
  13. 2023Separate And Diffuse: Using a Pretrained Diffusion Model for Improving Source Separationcited by 14
  14. 2024Annealed Multiple Choice Learning: Overcoming limitations of Winner-takes-all with annealingcited by 12
  15. 2022Resource-Efficient Separation Transformercited by 11
  16. 2023IIANet: An Intra- and Inter-Modality Attention Network for Audio-Visual Speech Separationcited by 11
  17. 2023Semantic Hearing: Programming Acoustic Scenes with Binaural Hearablescited by 11
  18. 2024Cross-Domain Audio Deepfake Detection: Dataset and Analysiscited by 11
  19. 2022Efficient Transformer-based Speech Enhancement Using Long Frames and STFT Magnitudescited by 10
  20. 2023Unifying Speech Enhancement and Separation with Gradient Modulation for End-to-End Noise-Robust Speech Separationcited by 10
  21. 2023Conformer-based Target-Speaker Automatic Speech Recognition for Single-Channel Audiocited by 10
  22. 2023ReZero: Region-customizable Sound Extractioncited by 10
  23. 2024TSELM: Target Speaker Extraction using Discrete Tokens and Language Modelscited by 10
  24. 2024TIGER: Time-frequency Interleaved Gain Extraction and Reconstruction for Efficient Speech Separationcited by 10
  25. 2022An Audio-Visual Speech Separation Model Inspired by Cortico-Thalamo-Cortical Circuitscited by 9
  26. 2024Noise-robust Speech Separation with Fast Generative Correctioncited by 9
  27. 2025SoloSpeech: Enhancing Intelligibility and Quality in Target Speech Extraction through a Cascaded Generative Pipelinecited by 9
  28. 2021Separate but Together: Unsupervised Federated Learning for Speech Enhancement from Non-IID Datacited by 8
  29. 2023A Generalized Bandsplit Neural Network for Cinematic Audio Source Separationcited by 8
  30. 2024OmniSep: Unified Omni-Modality Sound Separation with Query-Mixupcited by 8
  31. 2021Configurable Privacy-Preserving Automatic Speech Recognitioncited by 7
  32. 2021REAL-M: Towards Speech Separation on Real Mixturescited by 7
  33. 2022MixCycle: Unsupervised Speech Separation via Cyclic Mixture Permutation Invariant Trainingcited by 7
  34. 2024Separate in the Speech Chain: Cross-Modal Conditional Audio-Visual Target Speech Extractioncited by 7
  35. 2024AACessTalk: Fostering Communication between Minimally Verbal Autistic Children and Parents with Contextual Guidance and Card Recommendationcited by 7
  36. 2025Speech Enhancement Using Continuous Embeddings of Neural Audio Codeccited by 7
  37. 2021A Survey of Deep Learning for Low-Shot Object Detectioncited by 6
  38. 2022Latent Iterative Refinement for Modular Source Separationcited by 6

50 most cited of 184 All 184 (JSON)