# An Investigation on Speaker Augmentation for End-to-End Speaker Extraction

arXiv: https://arxiv.org/abs/2505.21805 · HTML: https://mlsanity.com/paper/2505.21805/

## Key facts

- Cited by 2 papers in the map (data through 19 Oct 2026).
- First posted on arXiv 27 May 2025.

## Built on by (most cited first, 2 of 2)

- [Lightweight speech enhancement guided target speech extraction in noisy multi-speaker scenarios](https://mlsanity.com/paper/2508.19583/) (2025)
- [TripleC Learning and Lightweight Speech Enhancement for Multi-Condition Target Speech Extraction](https://mlsanity.com/paper/2512.04945/) (2025)

## Abstract (arXiv)

Target confusion, defined as occasional switching to non-target speakers, poses a key challenge for end-to-end speaker extraction (E2E-SE) systems. We argue that this problem is largely caused by the lack of generalizability and discrimination of the speaker embeddings, and introduce a simple yet effective speaker augmentation strategy to tackle the problem. Specifically, we propose a time-domain resampling and rescaling pipeline that alters speaker traits while preserving other speech properties. This generates a variety of pseudo-speakers to help establish a generalizable speaker embedding space, while the speaker-trait-specific augmentation creates hard samples that force the model to focus on genuine speaker characteristics. Experiments on WSJ0-2Mix and LibriMix show that our method mitigates the target confusion and improves extraction performance. Moreover, it can be combined with metric learning, another effective approach to address target confusion, leading to further gains.