# Improving speaker discrimination of target speech extraction with time-domain SpeakerBeam

arXiv: https://arxiv.org/abs/2001.08378 · HTML: https://mlsanity.com/paper/2001.08378/

## Key facts

- Cited by 23 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Jan 2020.

## Built on by (most cited first, 23 of 23)

- [ESPnet-SPK: full pipeline speaker embedding toolkit with reproducible recipes, self-supervised front-ends, and off-the-shelf models](https://mlsanity.com/paper/2401.17230/) (2024)
- [Target Speech Extraction with Conditional Diffusion Model](https://mlsanity.com/paper/2308.03987/) (2023)
- [Multimodal Attention Fusion for Target Speaker Extraction](https://mlsanity.com/paper/2102.01326/) (2021)
- [SoloSpeech: Enhancing Intelligibility and Quality in Target Speech Extraction through a Cascaded Generative Pipeline](https://mlsanity.com/paper/2505.19314/) (2025)
- [X-SepFormer: End-to-end Speaker Extraction Network with Explicit Optimization on Speaker Confusion](https://mlsanity.com/paper/2303.05023/) (2023)
- [Typing to Listen at the Cocktail Party: Text-Guided Target Speaker Extraction](https://mlsanity.com/paper/2310.07284/) (2023)
- [Speaker Extraction with Co-Speech Gestures Cue](https://mlsanity.com/paper/2203.16840/) (2022)
- [Quantitative Evidence on Overlooked Aspects of Enrollment Speaker Embeddings for Target Speaker Separation](https://mlsanity.com/paper/2210.12635/) (2022)
- [TS-SUPERB: A Target Speech Processing Benchmark for Speech Self-Supervised Learning Models](https://mlsanity.com/paper/2505.06660/) (2025)
- [MeanFlow-TSE: One-Step Generative Target Speaker Extraction with Mean Flow](https://mlsanity.com/paper/2512.18572/) (2025)
- [Deformable Temporal Convolutional Networks for Monaural Noisy Reverberant Speech Separation](https://mlsanity.com/paper/2210.15305/) (2022)
- [Knowledge Distillation for Neural Transducer-based Target-Speaker ASR: Exploiting Parallel Mixture/Single-Talker Speech Data](https://mlsanity.com/paper/2305.15971/) (2023)
- [Adaptive Deterministic Flow Matching for Target Speaker Extraction](https://mlsanity.com/paper/2510.16995/) (2025)
- [Time-Domain Speech Extraction with Spatial Information and Multi Speaker Conditioning Mechanism](https://mlsanity.com/paper/2102.03762/) (2021)
- [Two-stage Framework for Robust Speech Emotion Recognition Using Target Speaker Extraction in Human Speech Noise Conditions](https://mlsanity.com/paper/2409.19585/) (2024)
- [Investigation of Speaker Representation for Target-Speaker Speech Processing](https://mlsanity.com/paper/2410.11243/) (2024)
- [Contextual Speech Extraction: Leveraging Textual History as an Implicit Cue for Target Speech Extraction](https://mlsanity.com/paper/2503.08798/) (2025)
- [Single microphone speaker extraction using unified time-frequency Siamese-Unet](https://mlsanity.com/paper/2203.02941/) (2022)
- [Analysis of impact of emotions on target speech extraction and speech separation](https://mlsanity.com/paper/2208.07091/) (2022)
- [Breaking the trade-off in personalized speech enhancement with cross-task knowledge distillation](https://mlsanity.com/paper/2211.02944/) (2022)
- [Improving Target Speaker Extraction with Sparse LDA-transformed Speaker Embeddings](https://mlsanity.com/paper/2301.06277/) (2023)
- [End-to-End Multi-Microphone Speaker Extraction Using Relative Transfer Functions](https://mlsanity.com/paper/2502.06285/) (2025)
- [Contrastive Knowledge Distillation for Embedding Refinement in Personalized Speech Enhancement](https://mlsanity.com/paper/2601.16235/) (2026)

## Abstract (arXiv)

Target speech extraction, which extracts a single target source in a mixture given clues about the target speaker, has attracted increasing attention. We have recently proposed SpeakerBeam, which exploits an adaptation utterance of the target speaker to extract his/her voice characteristics that are then used to guide a neural network towards extracting speech of that speaker. SpeakerBeam presents a practical alternative to speech separation as it enables tracking speech of a target speaker across utterances, and achieves promising speech extraction performance. However, it sometimes fails when speakers have similar voice characteristics, such as in same-gender mixtures, because it is difficult to discriminate the target speaker from the interfering speakers. In this paper, we investigate strategies for improving the speaker discrimination capability of SpeakerBeam. First, we propose a time-domain implementation of SpeakerBeam similar to that proposed for a time-domain audio separation network (TasNet), which has achieved state-of-the-art performance for speech separation. Besides, we investigate (1) the use of spatial features to better discriminate speakers when microphone array recordings are available, (2) adding an auxiliary speaker identification loss for helping to learn more discriminative voice characteristics. We show experimentally that these strategies greatly improve speech extraction performance, especially for same-gender mixtures, and outperform TasNet in terms of target speech extraction.