# Efficient Transformer-based Speech Enhancement Using Long Frames and STFT Magnitudes

arXiv: https://arxiv.org/abs/2206.11703 · HTML: https://mlsanity.com/paper/2206.11703/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 30 references that are in the map.
- First posted on arXiv 23 Jun 2022.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [WaveNet: A Generative Model for Raw Audio](https://mlsanity.com/paper/1609.03499/) (2016)
- [SDR - half-baked or well done?](https://mlsanity.com/paper/1811.02508/) (2018)
- [Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation](https://mlsanity.com/paper/1809.07454/) (2018)
- [Attention is All You Need in Speech Separation](https://mlsanity.com/paper/2010.13154/) (2020)
- [Dual-path RNN: efficient long sequence modeling for time-domain single-channel speech separation](https://mlsanity.com/paper/1910.06379/) (2019)
- [The INTERSPEECH 2020 Deep Noise Suppression Challenge: Datasets, Subjective Testing Framework, and Challenge Results](https://mlsanity.com/paper/2005.13981/) (2020)
- [Dual-Path Transformer Network: Direct Context-Aware Modeling for End-to-End Monaural Speech Separation](https://mlsanity.com/paper/2007.13975/) (2020)
- [T-GSA: Transformer with Gaussian-weighted self-attention for speech enhancement](https://mlsanity.com/paper/1910.06762/) (2019)
- [Filterbank design for end-to-end speech separation](https://mlsanity.com/paper/1910.10400/) (2019)
- [Demystifying TasNet: A Dissecting Approach](https://mlsanity.com/paper/1911.08895/) (2019)
- [A Multi-Phase Gammatone Filterbank for Speech Separation via TasNet](https://mlsanity.com/paper/1910.11615/) (2019)
- [Stabilizing Label Assignment for Speech Separation by Self-supervised Pre-training](https://mlsanity.com/paper/2010.15366/) (2020)
- [Phase-Aware Deep Speech Enhancement: It's All About The Frame Length](https://mlsanity.com/paper/2203.16222/) (2022)
- [SNR-Based Features and Diverse Training Data for Robust DNN-Based Speech Enhancement](https://mlsanity.com/paper/2004.03512/) (2020)

## Built on by (most cited first, 10 of 10)

- [Speech Enhancement Using Continuous Embeddings of Neural Audio Codec](https://mlsanity.com/paper/2502.16240/) (2025)
- [On the Behavior of Intrusive and Non-intrusive Speech Enhancement Metrics in Predictive and Generative Settings](https://mlsanity.com/paper/2306.03014/) (2023)
- [xLSTM-SENet: xLSTM for Single-Channel Speech Enhancement](https://mlsanity.com/paper/2501.06146/) (2025)
- [Deep neural network techniques for monaural speech enhancement: state of the art analysis](https://mlsanity.com/paper/2212.00369/) (2022)
- [Linguistic Knowledge Transfer Learning for Speech Enhancement](https://mlsanity.com/paper/2503.07078/) (2025)
- [MambAttention: Mamba with Multi-Head Attention for Generalizable Single-Channel Speech Enhancement](https://mlsanity.com/paper/2507.00966/) (2025)
- [ACA-Net: Towards Lightweight Speaker Verification using Asymmetric Cross Attention](https://mlsanity.com/paper/2305.12121/) (2023)
- [Effective Noise-aware Data Simulation for Domain-adaptive Speech Enhancement Leveraging Dynamic Stochastic Perturbation](https://mlsanity.com/paper/2409.01545/) (2024)
- [Active Speech Enhancement: Active Speech Denoising Decliping and Deveraberation](https://mlsanity.com/paper/2505.16911/) (2025)
- [Universal Robust Speech Adaptation for Cross-Domain Speech Recognition and Enhancement](https://mlsanity.com/paper/2602.04307/) (2026)

## Abstract (arXiv)

The SepFormer architecture shows very good results in speech separation. Like other learned-encoder models, it uses short frames, as they have been shown to obtain better performance in these cases. This results in a large number of frames at the input, which is problematic; since the SepFormer is transformer-based, its computational complexity drastically increases with longer sequences. In this paper, we employ the SepFormer in a speech enhancement task and show that by replacing the learned-encoder features with a magnitude short-time Fourier transform (STFT) representation, we can use long frames without compromising perceptual enhancement performance. We obtained equivalent quality and intelligibility evaluation scores while reducing the number of operations by a factor of approximately 8 for a 10-second utterance.