# Speech Enhancement with Overlapped-Frame Information Fusion and Causal Self-Attention

arXiv: https://arxiv.org/abs/2501.12004 · HTML: https://mlsanity.com/paper/2501.12004/

## Key facts

- No citing papers in the map yet (data through 19 Oct 2026).
- Placed in Speech enhancement and noise suppression, one of 11 directions in Speech and audio language models.
- Builds on 19 of its 28 references that are in the map.
- First posted on arXiv 21 Jan 2025.

## Placed in

- Speech and audio language models / [Speech enhancement and noise suppression](https://mlsanity.com/speech/speech-enhancement-and-noise-suppression/)

## Builds on (most cited first, 19 of 19)

- [SDR - half-baked or well done?](https://mlsanity.com/paper/1811.02508/) (2018)
- [DCCRN: Deep Complex Convolution Recurrent Network for Phase-Aware Speech Enhancement](https://mlsanity.com/paper/2008.00264/) (2020)
- [Real Time Speech Enhancement in the Waveform Domain](https://mlsanity.com/paper/2006.12847/) (2020)
- [Dual-path RNN: efficient long sequence modeling for time-domain single-channel speech separation](https://mlsanity.com/paper/1910.06379/) (2019)
- [SEGAN: Speech Enhancement Generative Adversarial Network](https://mlsanity.com/paper/1703.09452/) (2017)
- [The INTERSPEECH 2020 Deep Noise Suppression Challenge: Datasets, Subjective Testing Framework, and Challenge Results](https://mlsanity.com/paper/2005.13981/) (2020)
- [FullSubNet: A Full-Band and Sub-Band Fusion Model for Real-Time Single-Channel Speech Enhancement](https://mlsanity.com/paper/2010.15508/) (2020)
- [CMGAN: Conformer-based Metric GAN for Speech Enhancement](https://mlsanity.com/paper/2203.15149/) (2022)
- [FRCRN: Boosting Feature Representation using Frequency Recurrence for Monaural Speech Enhancement](https://mlsanity.com/paper/2206.07293/) (2022)
- [Glance and Gaze: A Collaborative Learning Framework for Single-channel Speech Enhancement](https://mlsanity.com/paper/2106.11789/) (2021)
- [PoCoNet: Better Speech Enhancement with Frequency-Positional Embeddings, Semi-Supervised Conversational Data, and Biased Loss](https://mlsanity.com/paper/2008.04470/) (2020)
- [FullSubNet+: Channel Attention FullSubNet with Complex Spectrograms for Speech Enhancement](https://mlsanity.com/paper/2203.12188/) (2022)
- [Inter-SubNet: Speech Enhancement with Subband Interaction](https://mlsanity.com/paper/2305.05599/) (2023)
- [Real-time Monaural Speech Enhancement With Short-time Discrete Cosine Transform](https://mlsanity.com/paper/2102.04629/) (2021)
- [Improving Frame-Online Neural Speech Enhancement with Overlapped-Frame Prediction](https://mlsanity.com/paper/2204.07566/) (2022)
- [SICRN: Advancing Speech Enhancement through State Space Model and Inplace Convolution Techniques](https://mlsanity.com/paper/2402.14225/) (2024)
- [A Two-Stage Framework in Cross-Spectrum Domain for Real-Time Speech Enhancement](https://mlsanity.com/paper/2401.10494/) (2024)
- [Speech Enhancement with Fullband-Subband Cross-Attention Network](https://mlsanity.com/paper/2211.05432/) (2022)
- [VSANet: Real-time Speech Enhancement Based on Voice Activity Detection and Causal Spatial Attention](https://mlsanity.com/paper/2310.07295/) (2023)

## Abstract (arXiv)

For time-frequency (TF) domain speech enhancement (SE) methods, the overlap-and-add operation in the inverse TF transformation inevitably leads to an algorithmic delay equal to the window size. However, typical causal SE systems fail to utilize the future speech information within this inherent delay, thereby limiting SE performance. In this paper, we propose an overlapped-frame information fusion scheme. At each frame index, we construct several pseudo overlapped-frames, fuse them with the original speech frame, and then send the fused results to the SE model. Additionally, we introduce a causal time-frequency-channel attention (TFCA) block to boost the representation capability of the neural network. This block parallelly processes the intermediate feature maps through self-attention-based operations in the time, frequency, and channel dimensions. Experiments demonstrate the superiority of these improvements, and the proposed SE system outperforms the current advanced methods.