# Target Speaker Extraction through Comparing Noisy Positive and Negative Audio Enrollments

arXiv: https://arxiv.org/abs/2502.16611 · HTML: https://mlsanity.com/paper/2502.16611/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 39 references that are in the map.
- First posted on arXiv 23 Feb 2025.

## Builds on (most cited first, 19 of 19)

- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [An Empirical Study of Training Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.02057/) (2021)
- [WHAM!: Extending Speech Separation to Noisy Environments](https://mlsanity.com/paper/1907.01160/) (2019)
- [VoiceFilter: Targeted Voice Separation by Speaker-Conditioned Spectrogram Masking](https://mlsanity.com/paper/1810.04826/) (2018)
- [TF-GridNet: Making Time-Frequency Domain Models Great Again for Monaural Speaker Separation](https://mlsanity.com/paper/2209.03952/) (2022)
- [SpEx+: A Complete Time Domain Speaker Extraction Network](https://mlsanity.com/paper/2005.04686/) (2020)
- [SpEx: Multi-Scale Time Domain Speaker Extraction Network](https://mlsanity.com/paper/2004.08326/) (2020)
- [Muse: Multi-modal target speaker extraction with visual cues](https://mlsanity.com/paper/2010.07775/) (2020)
- [Selective Listening by Synchronizing Speech with Lips](https://mlsanity.com/paper/2106.07150/) (2021)
- [Look Once to Hear: Target Speech Hearing with Noisy Examples](https://mlsanity.com/paper/2405.06289/) (2024)
- [Typing to Listen at the Cocktail Party: Text-Guided Target Speaker Extraction](https://mlsanity.com/paper/2310.07284/) (2023)
- [Self-Supervised Disentangled Representation Learning for Robust Target Speech Extraction](https://mlsanity.com/paper/2312.10305/) (2023)
- [Multi-user VoiceFilter-Lite via Attentive Speaker Embedding](https://mlsanity.com/paper/2107.01201/) (2021)
- [Target conversation extraction: Source separation using turn-taking dynamics](https://mlsanity.com/paper/2407.11277/) (2024)
- [Speaker activity driven neural speech extraction](https://mlsanity.com/paper/2101.05516/) (2021)
- [Multi-Level Speaker Representation for Target Speaker Extraction](https://mlsanity.com/paper/2410.16059/) (2024)
- [Hierarchical speaker representation for target speaker extraction](https://mlsanity.com/paper/2210.15849/) (2022)
- [Continuous Target Speech Extraction: Enhancing Personalized Diarization and Extraction on Complex Recordings](https://mlsanity.com/paper/2401.15993/) (2024)
- [Binaural Selective Attention Model for Target Speaker Extraction](https://mlsanity.com/paper/2406.12236/) (2024)

## Built on by (most cited first, 1 of 1)

- [Shared Representation Learning for Reference-Guided Targeted Sound Detection](https://mlsanity.com/paper/2603.17025/) (2026)

## Abstract (arXiv)

Target speaker extraction focuses on isolating a specific speaker's voice from an audio mixture containing multiple speakers. To provide information about the target speaker's identity, prior works have utilized clean audio samples as conditioning inputs. However, such clean audio examples are not always readily available. For instance, obtaining a clean recording of a stranger's voice at a cocktail party without leaving the noisy environment is generally infeasible. Limited prior research has explored extracting the target speaker's characteristics from noisy enrollments, which may contain overlapping speech from interfering speakers. In this work, we explore a novel enrollment strategy that encodes target speaker information from the noisy enrollment by comparing segments where the target speaker is talking (Positive Enrollments) with segments where the target speaker is silent (Negative Enrollments). Experiments show the effectiveness of our model architecture, which achieves over 2.1 dB higher SI-SNRi compared to prior works in extracting the monaural speech from the mixture of two speakers. Additionally, the proposed two-stage training strategy accelerates convergence, reducing the number of optimization steps required to reach 3 dB SNR by 60%. Overall, our method achieves state-of-the-art performance in the monaural target speaker extraction conditioned on noisy enrollments. Our implementation is available at https://github.com/xu-shitong/TSE-through-Positive-Negative-Enroll .