# Guided Speech Enhancement Network

arXiv: https://arxiv.org/abs/2303.07486 · HTML: https://mlsanity.com/paper/2303.07486/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 32 references that are in the map.
- First posted on arXiv 13 Mar 2023.

## Builds on (most cited first, 14 of 14)

- [Conv-TasNet: Surpassing Ideal Time-Frequency Magnitude Masking for Speech Separation](https://mlsanity.com/paper/1809.07454/) (2018)
- [DDSP: Differentiable Digital Signal Processing](https://mlsanity.com/paper/2001.04643/) (2020)
- [DCCRN: Deep Complex Convolution Recurrent Network for Phase-Aware Speech Enhancement](https://mlsanity.com/paper/2008.00264/) (2020)
- [NU-Wave: A Diffusion Probabilistic Model for Neural Audio Upsampling](https://mlsanity.com/paper/2104.02321/) (2021)
- [Streaming keyword spotting on mobile devices](https://mlsanity.com/paper/2005.06720/) (2020)
- [VoiceFixer: A Unified Framework for High-Fidelity Speech Restoration](https://mlsanity.com/paper/2204.05841/) (2022)
- [End-to-end Microphone Permutation and Number Invariant Multi-channel Speech Separation](https://mlsanity.com/paper/1910.14104/) (2019)
- [FaSNet: Low-latency Adaptive Beamforming for Multi-microphone Audio Processing](https://mlsanity.com/paper/1909.13387/) (2019)
- [Real-time Speech Frequency Bandwidth Extension](https://mlsanity.com/paper/2010.10677/) (2020)
- [ADL-MVDR: All deep learning MVDR beamformer for target speech separation](https://mlsanity.com/paper/2008.06994/) (2020)
- [A Simultaneous Denoising and Dereverberation Framework with Target Decoupling](https://mlsanity.com/paper/2106.12743/) (2021)
- [Embedding and Beamforming: All-neural Causal Beamformer for Multichannel Speech Enhancement](https://mlsanity.com/paper/2109.00265/) (2021)
- [Learning to Denoise Historical Music](https://mlsanity.com/paper/2008.02027/) (2020)
- [A Universally-Deployable ASR Frontend for Joint Acoustic Echo Cancellation, Speech Enhancement, and Voice Separation](https://mlsanity.com/paper/2209.06410/) (2022)

## Built on by (most cited first, 1 of 1)

- [Binaural Angular Separation Network](https://mlsanity.com/paper/2401.08864/) (2024)

## Abstract (arXiv)

High quality speech capture has been widely studied for both voice communication and human computer interface reasons. To improve the capture performance, we can often find multi-microphone speech enhancement techniques deployed on various devices. Multi-microphone speech enhancement problem is often decomposed into two decoupled steps: a beamformer that provides spatial filtering and a single-channel speech enhancement model that cleans up the beamformer output. In this work, we propose a speech enhancement solution that takes both the raw microphone and beamformer outputs as the input for an ML model. We devise a simple yet effective training scheme that allows the model to learn from the cues of the beamformer by contrasting the two inputs and greatly boost its capability in spatial rejection, while conducting the general tasks of denoising and dereverberation. The proposed solution takes advantage of classical spatial filtering algorithms instead of competing with them. By design, the beamformer module then could be selected separately and does not require a large amount of data to be optimized for a given form factor, and the network model can be considered as a standalone module which is highly transferable independently from the microphone array. We name the ML module in our solution as GSENet, short for Guided Speech Enhancement Network. We demonstrate its effectiveness on real world data collected on multi-microphone devices in terms of the suppression of noise and interfering speech.