# Glance and Gaze: A Collaborative Learning Framework for Single-channel Speech Enhancement

arXiv: https://arxiv.org/abs/2106.11789 · HTML: https://mlsanity.com/paper/2106.11789/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Jun 2021.

## Built on by (most cited first, 25 of 25)

- [Speech Enhancement and Dereverberation with Diffusion-based Generative Models](https://mlsanity.com/paper/2208.05830/) (2022)
- [StoRM: A Diffusion-based Stochastic Regeneration Model for Speech Enhancement and Dereverberation](https://mlsanity.com/paper/2212.11851/) (2022)
- [CMGAN: Conformer-based Metric GAN for Speech Enhancement](https://mlsanity.com/paper/2203.15149/) (2022)
- [DeepFilterNet2: Towards Real-Time Speech Enhancement on Embedded Devices for Full-Band Audio](https://mlsanity.com/paper/2205.05474/) (2022)
- [CMGAN: Conformer-Based Metric-GAN for Monaural Speech Enhancement](https://mlsanity.com/paper/2209.11112/) (2022)
- [Spiking Structured State Space Model for Monaural Speech Enhancement](https://mlsanity.com/paper/2309.03641/) (2023)
- [Dual-branch Attention-In-Attention Transformer for single-channel speech enhancement](https://mlsanity.com/paper/2110.06467/) (2021)
- [Diffusion-Based Speech Enhancement with Joint Generative and Predictive Decoders](https://mlsanity.com/paper/2305.10734/) (2023)
- [McNet: Fuse Multiple Cues for Multichannel Speech Enhancement](https://mlsanity.com/paper/2211.08872/) (2022)
- [Noise-aware Speech Enhancement using Diffusion Probabilistic Model](https://mlsanity.com/paper/2307.08029/) (2023)
- [Thunder : Unified Regression-Diffusion Speech Enhancement with a Single Reverse Step using Brownian Bridge](https://mlsanity.com/paper/2406.06139/) (2024)
- [SonicSim: A customizable simulation platform for speech processing in moving sound source scenarios](https://mlsanity.com/paper/2410.01481/) (2024)
- [Wind Noise Reduction with a Diffusion-based Stochastic Regeneration Model](https://mlsanity.com/paper/2306.12867/) (2023)
- [Extending DNN-based Multiplicative Masking to Deep Subband Filtering for Improved Dereverberation](https://mlsanity.com/paper/2303.00529/) (2023)
- [CleanUNet 2: A Hybrid Speech Denoising Model on Waveform and Spectrogram](https://mlsanity.com/paper/2309.05975/) (2023)
- [ROSE: A Recognition-Oriented Speech Enhancement Framework in Air Traffic Control Using Multi-Objective Learning](https://mlsanity.com/paper/2312.06118/) (2023)
- [CleanMel: Mel-Spectrogram Enhancement for Improving Both Speech Quality and ASR](https://mlsanity.com/paper/2502.20040/) (2025)
- [A Two-Stage Hierarchical Deep Filtering Framework for Real-Time Speech Enhancement](https://mlsanity.com/paper/2506.01023/) (2025)
- [A neural network-supported two-stage algorithm for lightweight dereverberation on hearing devices](https://mlsanity.com/paper/2204.02978/) (2022)
- [Efficient Monaural Speech Enhancement using Spectrum Attention Fusion](https://mlsanity.com/paper/2308.02263/) (2023)
- [ConcateNet: Dialogue Separation Using Local And Global Feature Concatenation](https://mlsanity.com/paper/2408.08729/) (2024)
- [UL-UNAS: Ultra-Lightweight U-Nets for Real-Time Speech Enhancement via Network Architecture Search](https://mlsanity.com/paper/2503.00340/) (2025)
- [Combining Deterministic Enhanced Conditions with Dual-Streaming Encoding for Diffusion-Based Speech Enhancement](https://mlsanity.com/paper/2505.13983/) (2025)
- [DBHN-Net: Dual-Branch Hybrid Neural Network For Low-Complexity Monaural Speech Enhancement](https://mlsanity.com/paper/2606.05911/) (2026)
- [Neuromorphic Speech Enhancement with Dual-Branch Spiking Neural Networks](https://mlsanity.com/paper/2606.23761/) (2026)

## Abstract (arXiv)

The capability of the human to pay attention to both coarse and fine-grained regions has been applied to computer vision tasks. Motivated by that, we propose a collaborative learning framework in the complex domain for monaural noise suppression. The proposed system consists of two principal modules, namely spectral feature extraction module (FEM) and stacked glance-gaze modules (GGMs). In FEM, the UNet-block is introduced after each convolution layer, enabling the feature recalibration from multiple scales. In each GGM, we decompose the multi-target optimization in the complex spectrum into two sub-tasks. Specifically, the glance path aims to suppress the noise in the magnitude domain to obtain a coarse estimation, and meanwhile, the gaze path attempts to compensate for the lost spectral detail in the complex domain. The two paths work collaboratively and facilitate spectral estimation from complementary perspectives. Besides, by repeatedly unfolding the GGMs, the intermediate result can be iteratively refined across stages and lead to the ultimate estimation of the spectrum. The experiments are conducted on the WSJ0-SI84, DNS-Challenge dataset, and Voicebank+Demand dataset. Results show that the proposed approach achieves state-of-the-art performance over previous advanced systems on the WSJ0-SI84 and DNS-Challenge dataset, and meanwhile, competitive performance is achieved on the Voicebank+Demand corpus.