# Lightweight speech enhancement guided target speech extraction in noisy multi-speaker scenarios

arXiv: https://arxiv.org/abs/2508.19583 · HTML: https://mlsanity.com/paper/2508.19583/

## Key facts

- Cited by 1 papers in the map (data through 19 Oct 2026).
- Placed in Speech enhancement and noise suppression, one of 11 directions in Speech and audio language models.
- Builds on 22 of its 32 references that are in the map.
- First posted on arXiv 27 Aug 2025.

## Placed in

- Speech and audio language models / [Speech enhancement and noise suppression](https://mlsanity.com/speech/speech-enhancement-and-noise-suppression/)

## Builds on (most cited first, 22 of 22)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [ECAPA-TDNN: Emphasized Channel Attention, Propagation and Aggregation in TDNN Based Speaker Verification](https://mlsanity.com/paper/2005.07143/) (2020)
- [SDR - half-baked or well done?](https://mlsanity.com/paper/1811.02508/) (2018)
- [LibriMix: An Open-Source Dataset for Generalizable Speech Separation](https://mlsanity.com/paper/2005.11262/) (2020)
- [SpEx+: A Complete Time Domain Speaker Extraction Network](https://mlsanity.com/paper/2005.04686/) (2020)
- [SpEx: Multi-Scale Time Domain Speaker Extraction Network](https://mlsanity.com/paper/2004.08326/) (2020)
- [Improving speaker discrimination of target speech extraction with time-domain SpeakerBeam](https://mlsanity.com/paper/2001.08378/) (2020)
- [DeepFilterNet: Perceptually Motivated Real-Time Speech Enhancement](https://mlsanity.com/paper/2305.08227/) (2023)
- [High Fidelity Speech Enhancement with Band-split RNN](https://mlsanity.com/paper/2212.00406/) (2022)
- [X-SepFormer: End-to-end Speaker Extraction Network with Explicit Optimization on Speaker Confusion](https://mlsanity.com/paper/2303.05023/) (2023)
- [TEA-PSE 3.0: Tencent-Ethereal-Audio-Lab Personalized Speech Enhancement System For ICASSP 2023 DNS Challenge](https://mlsanity.com/paper/2303.07704/) (2023)
- [DPCCN: Densely-Connected Pyramid Complex Convolutional Network for Robust Speech Separation And Extraction](https://mlsanity.com/paper/2112.13520/) (2021)
- [Target Speaker Extraction by Directly Exploiting Contextual Information in the Time-Frequency Domain](https://mlsanity.com/paper/2402.17146/) (2024)
- [Personalized Speech Enhancement Without a Separate Speaker Embedding Model](https://mlsanity.com/paper/2406.09928/) (2024)
- [Hierarchical speaker representation for target speaker extraction](https://mlsanity.com/paper/2210.15849/) (2022)
- [MC-SpEx: Towards Effective Speaker Extraction with Multi-Scale Interfusion and Conditional Speaker Modulation](https://mlsanity.com/paper/2306.16250/) (2023)
- [Multi-Level Speaker Representation for Target Speaker Extraction](https://mlsanity.com/paper/2410.16059/) (2024)
- [SEF-PNet: Speaker Encoder-Free Personalized Speech Enhancement with Local and Global Contexts Aggregation](https://mlsanity.com/paper/2501.11274/) (2025)
- [3S-TSE: Efficient Three-Stage Target Speaker Extraction for Real-Time and Low-Resource Applications](https://mlsanity.com/paper/2312.10979/) (2023)
- [A Two-Stage Framework in Cross-Spectrum Domain for Real-Time Speech Enhancement](https://mlsanity.com/paper/2401.10494/) (2024)
- [RaD-Net 2: A causal two-stage repairing and denoising speech enhancement network with knowledge distillation and complex axial self-attention](https://mlsanity.com/paper/2406.07498/) (2024)
- [An Investigation on Speaker Augmentation for End-to-End Speaker Extraction](https://mlsanity.com/paper/2505.21805/) (2025)

## Built on by (most cited first, 1 of 1)

- [TripleC Learning and Lightweight Speech Enhancement for Multi-Condition Target Speech Extraction](https://mlsanity.com/paper/2512.04945/) (2025)

## Abstract (arXiv)

Target speech extraction (TSE) has achieved strong performance in relatively simple conditions such as one-speaker-plus-noise and two-speaker mixtures, but its performance remains unsatisfactory in noisy multi-speaker scenarios. To address this issue, we introduce a lightweight speech enhancement model, GTCRN, to better guide TSE in noisy environments. Building on our competitive previous speaker embedding/encoder-free framework SEF-PNet, we propose two extensions: LGTSE and D-LGTSE. LGTSE incorporates noise-agnostic enrollment guidance by denoising the input noisy speech before context interaction with enrollment speech, thereby reducing noise interference. D-LGTSE further improves system robustness against speech distortion by leveraging denoised speech as an additional noisy input during training, expanding the dynamic range of noisy conditions and enabling the model to directly learn from distorted signals. Furthermore, we propose a two-stage training strategy, first with GTCRN enhancement-guided pre-training and then joint fine-tuning, to fully exploit model potential.Experiments on the Libri2Mix dataset demonstrate significant improvements of 0.89 dB in SISDR, 0.16 in PESQ, and 1.97% in STOI, validating the effectiveness of our approach.