mlsanityData to 19 Oct 2026

Improving Frame-Online Neural Speech Enhancement with Overlapped-Frame Prediction

arXiv 2204.07566 ↗First posted 15 Apr 2022cs.SD eess.AScited by 5

On the map: Speech and audio language models → Speech enhancement and noise suppression

Abstract (arXiv)

Frame-online speech enhancement systems in the short-time Fourier transform (STFT) domain usually have an algorithmic latency equal to the window size due to the use of overlap-add in the inverse STFT (iSTFT). This algorithmic latency allows the enhancement models to leverage future contextual information up to a length equal to the window size. However, this information is only partially leveraged by current frame-online systems. To fully exploit it, we propose an overlapped-frame prediction technique for deep learning based frame-online speech enhancement, where at each frame our deep neural network (DNN) predicts the current and several past frames that are necessary for overlap-add, instead of only predicting the current frame. In addition, we propose a loss function to account for the scale difference between predicted and oracle target signals. Experiments on a noisy-reverberant speech enhancement task show the effectiveness of the proposed algorithms.

Key facts. Cited by 5 papers in the map (data through 19 Oct 2026). Placed in Speech enhancement and noise suppression, one of 11 directions in Speech and audio language models. Builds on 24 of its 47 references that are in the map. First posted on arXiv 15 Apr 2022.

Builds on

24 of 47 references.

Closest ancestors · 4

Same direction, newest first

  1. 2022ICASSP 2022 Acoustic Echo Cancellation Challenge27 Feb
  2. 2021On The Compensation Between Magnitude and Phase in Speech Separation11 Aug
  3. 2021INTERSPEECH 2021 ConferencingSpeech Challenge: Towards Far-field Multi-Channel Speech Enhancement for Video Conferencing2 Apr
  4. 2021Towards efficient models for real-time deep noise suppression22 Jan

Other references · 20

  1. 2018SDR - half-baked or well done?cited by 626
  2. 2018Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separationcited by 596
  3. 2020FSD50K: An Open Dataset of Human-Labeled Sound Eventscited by 395
  4. 2017Supervised Speech Separation Based on Deep Learning: An Overviewcited by 342
  5. 2020DCCRN: Deep Complex Convolution Recurrent Network for Phase-Aware Speech Enhancementcited by 284
  6. 2020Real Time Speech Enhancement in the Waveform Domaincited by 282
  7. 2017TasNet: time-domain audio separation network for real-time, single-channel speech separationcited by 157
  8. 2020FullSubNet: A Full-Band and Sub-Band Fusion Model for Real-Time Single-Channel Speech Enhancementcited by 121
  9. 2018Differentiable Consistency Constraints for Improved Deep Speech Enhancementcited by 53
  10. 2020Weighted Speech Distortion Losses for Neural-network-based Real-time Speech Enhancementcited by 39
  11. 2020Multi-microphone Complex Spectral Mapping for Utterance-wise and Continuous Speech Separationcited by 35
  12. 2020The INTERSPEECH 2020 Deep Noise Suppression Challenge: Datasets, Subjective Speech Quality and Testing Frameworkcited by 30
8 more
  1. 2017Complex spectrogram enhancement by convolutional neural network with multi-metrics learningcited by 27
  2. 2020Real-time binaural speech separation with preserved spatial cuescited by 22
  3. 2021ICASSP 2021 Deep Noise Suppression Challenge: Decoupling Magnitude and Phase Optimization with a Two-Stage Deep Networkcited by 21
  4. 2021Convolutive Prediction for Monaural Speech Dereverberation and Noisy-Reverberant Speaker Separationcited by 21
  5. 2022SkiM: Skipping Memory LSTM for Low-Latency Real-Time Continuous Speech Separationcited by 20
  6. 2018Exploring Tradeoffs in Models for Low-latency Speech Enhancementcited by 11
  7. 2021Leveraging Low-Distortion Target Estimates for Improved Speech Enhancementcited by 10
  8. 2019Low-Latency Speaker-Independent Continuous Speech Separationcited by 9

Built on by

Cited by year (2026 to 19 Oct)

Citing papers per year: 2023: 2; 2024: 1; 2025: 1; 2026: 1.

5 papers.

  1. 2024DualVC 3: Leveraging Language Model Generated Pseudo Context for End-to-end Low Latency Streaming Voice Conversioncited by 6
  2. 2023End-to-End Integration of Speech Separation and Voice Activity Detection for Low-Latency Diarization of Telephone Conversationscited by 2
  3. 2023Causal Signal-Based DCCRN with Overlapped-Frame Prediction for Online Speech Enhancementcited by 2
  4. 2025Speech Enhancement with Overlapped-Frame Information Fusion and Causal Self-Attentioncited by 0
  5. 2026HALO: Half-Frame-Rate Adaptive Learnable Operator for Lightweight STFT-Based Speech Enhancementcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022STFT-Domain Neural Speech Enhancement with Very Low Algorithmic Latency
  2. 2022Phase-Aware Deep Speech Enhancement: It's All About The Frame Length
  3. 2022Efficient Transformer-based Speech Enhancement Using Long Frames and STFT Magnitudes
  4. 2022DNN-Free Low-Latency Adaptive Speech Enhancement Based on Frame-Online Beamforming Powered by Block-Online FastMNMF
  5. 2024Modulating State Space Model with SlowFast Framework for Compute-Efficient Ultra Low-Latency Speech Enhancement
  6. 2022Learning to Enhance or Not: Neural Network-Based Switching of Enhanced and Observed Signals for Overlapping Speech Recognition
  7. 2021A Modulation-Domain Loss for Neural-Network-based Real-time Speech Enhancement
  8. 2025ARiSE: Auto-Regressive Multi-Channel Speech Enhancement