On real-time multi-stage speech enhancement systems
On the map: Speech and audio language models → Speech enhancement and noise suppression
Abstract (arXiv)
Recently, multi-stage systems have stood out among deep learning-based speech enhancement methods. However, these systems are always high in complexity, requiring millions of parameters and powerful computational resources, which limits their application for real-time processing in low-power devices. Besides, the contribution of various influencing factors to the success of multi-stage systems remains unclear, which presents challenges to reduce the size of these systems. In this paper, we extensively investigate a lightweight two-stage network with only 560k total parameters. It consists of a Mel-scale magnitude masking model in the first stage and a complex spectrum mapping model in the second stage. We first provide a consolidated view of the roles of gain power factor, post-filter, and training labels for the Mel-scale masking model. Then, we explore several training schemes for the two-stage network and provide some insights into the superiority of the two-stage network. We show that the proposed two-stage network trained by an optimal scheme achieves a performance similar to a four times larger open source model DeepFilterNet2.
Key facts. No citing papers in the map yet (data through 19 Oct 2026). Placed in Speech enhancement and noise suppression, one of 11 directions in Speech and audio language models. Builds on 12 of its 24 references that are in the map. First posted on arXiv 19 Dec 2023.
Builds on
12 of 24 references.
Closest ancestors · 4
Same direction, newest first
- 2023EffCRN: An Efficient Convolutional Recurrent Network for High-Performance Speech Enhancement5 Jun
- 2023Deep Multi-Frame Filtering for Hearing Aids14 May
- 2023The NPU-Elevoc Personalized Speech Enhancement System for ICASSP2023 DNS Challenge13 Mar
- 2022DeepFilterNet2: Towards Real-Time Speech Enhancement on Embedded Devices for Full-Band Audio11 May
Other references · 8
- 2018SDR - half-baked or well done?cited by 626
- 2017Supervised Speech Separation Based on Deep Learning: An Overviewcited by 342
- 2020DCCRN: Deep Complex Convolution Recurrent Network for Phase-Aware Speech Enhancementcited by 284
- 2017A Hybrid DSP/Deep Learning Approach to Real-Time Full-Band Speech Enhancementcited by 67
- 2020A Perceptually-Motivated Approach for Low-Complexity, Real-Time Enhancement of Fullband Speechcited by 53
- 2020A consolidated view of loss functions for supervised deep learning-based speech enhancementcited by 43
- 2021ICASSP 2021 Deep Noise Suppression Challenge: Decoupling Magnitude and Phase Optimization with a Two-Stage Deep Networkcited by 21
- 2023DeepVQE: Real Time Deep Voice Quality Enhancement for Joint Acoustic Echo Cancellation, Noise Suppression and Dereverberationcited by 12
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2021Incorporating Multi-Target in Multi-Stage Speech Enhancement Model for Better Generalization
- 2021Speech Enhancement Using Multi-Stage Self-Attentive Temporal Convolutional Networks
- 2023THLNet: two-stage heterogeneous lightweight network for monaural speech enhancement
- 2024Dense-TSNet: Dense Connected Two-Stage Structure for Ultra-Lightweight Speech Enhancement
- 2025A Two-Stage Hierarchical Deep Filtering Framework for Real-Time Speech Enhancement
- 2021A Two-stage Complex Network using Cycle-consistent Generative Adversarial Networks for Speech Enhancement
- 2021Towards efficient models for real-time deep noise suppression
- 2024Beyond Performance Plateaus: A Comprehensive Study on Scalability in Speech Enhancement