# SpEx+: A Complete Time Domain Speaker Extraction Network

arXiv: https://arxiv.org/abs/2005.04686 · HTML: https://mlsanity.com/paper/2005.04686/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 May 2020.

## Built on by (most cited first, 30 of 34)

- [Target Speech Extraction with Conditional Diffusion Model](https://mlsanity.com/paper/2308.03987/) (2023)
- [Look Once to Hear: Target Speech Hearing with Noisy Examples](https://mlsanity.com/paper/2405.06289/) (2024)
- [TSELM: Target Speaker Extraction using Discrete Tokens and Language Models](https://mlsanity.com/paper/2409.07841/) (2024)
- [An Audio-Visual Speech Separation Model Inspired by Cortico-Thalamo-Cortical Circuits](https://mlsanity.com/paper/2212.10744/) (2022)
- [SoloSpeech: Enhancing Intelligibility and Quality in Target Speech Extraction through a Cascaded Generative Pipeline](https://mlsanity.com/paper/2505.19314/) (2025)
- [Separate in the Speech Chain: Cross-Modal Conditional Audio-Visual Target Speech Extraction](https://mlsanity.com/paper/2404.12725/) (2024)
- [X-SepFormer: End-to-end Speaker Extraction Network with Explicit Optimization on Speaker Confusion](https://mlsanity.com/paper/2303.05023/) (2023)
- [Typing to Listen at the Cocktail Party: Text-Guided Target Speaker Extraction](https://mlsanity.com/paper/2310.07284/) (2023)
- [FlowTSE: Target Speaker Extraction with Flow Matching](https://mlsanity.com/paper/2505.14465/) (2025)
- [Speaker Extraction with Co-Speech Gestures Cue](https://mlsanity.com/paper/2203.16840/) (2022)
- [Self-Supervised Disentangled Representation Learning for Robust Target Speech Extraction](https://mlsanity.com/paper/2312.10305/) (2023)
- [LauraTSE: Target Speaker Extraction using Auto-Regressive Decoder-Only Language Models](https://mlsanity.com/paper/2504.07402/) (2025)
- [Conditional Diffusion Model for Target Speaker Extraction](https://mlsanity.com/paper/2310.04791/) (2023)
- [UniSE: A Unified Framework for Decoder-Only Autoregressive LM-Based Speech Enhancement](https://mlsanity.com/paper/2510.20441/) (2025)
- [MeanFlow-TSE: One-Step Generative Target Speaker Extraction with Mean Flow](https://mlsanity.com/paper/2512.18572/) (2025)
- [WASE: Learning When to Attend for Speaker Extraction in Cocktail Party Environments](https://mlsanity.com/paper/2106.07016/) (2021)
- [Improving Audio-Visual Speech Recognition by Lip-Subword Correlation Based Visual Pre-training and Cross-Modal Fusion Encoder](https://mlsanity.com/paper/2308.08488/) (2023)
- [Adaptive Deterministic Flow Matching for Target Speaker Extraction](https://mlsanity.com/paper/2510.16995/) (2025)
- [Time-Domain Speech Extraction with Spatial Information and Multi Speaker Conditioning Mechanism](https://mlsanity.com/paper/2102.03762/) (2021)
- [Single microphone speaker extraction using unified time-frequency Siamese-Unet](https://mlsanity.com/paper/2203.02941/) (2022)
- [MBTFNet: Multi-Band Temporal-Frequency Neural Network For Singing Voice Enhancement](https://mlsanity.com/paper/2310.04369/) (2023)
- [TDFNet: An Efficient Audio-Visual Speech Separation Model with Top-down Fusion](https://mlsanity.com/paper/2401.14185/) (2024)
- [Target Speaker Extraction through Comparing Noisy Positive and Negative Audio Enrollments](https://mlsanity.com/paper/2502.16611/) (2025)
- [GenTSE: Enhancing Target Speaker Extraction via a Coarse-to-Fine Generative Language Model](https://mlsanity.com/paper/2512.20978/) (2025)
- [SAM Audio Judge: A Unified Multimodal Framework for Perceptual Evaluation of Audio Separation](https://mlsanity.com/paper/2601.19702/) (2026)
- [StarTSE: Towards Streaming Target Speaker Extraction via Chunk-wise Interleaved Splicing of Autoregressive Language Model](https://mlsanity.com/paper/2604.19635/) (2026)
- [Sparsely Overlapped Speech Training in the Time Domain: Joint Learning of Target Speech Separation and Personal VAD Benefits](https://mlsanity.com/paper/2106.14371/) (2021)
- [VCSE: Time-Domain Visual-Contextual Speaker Extraction Network](https://mlsanity.com/paper/2210.06177/) (2022)
- [Improving Target Speaker Extraction with Sparse LDA-transformed Speaker Embeddings](https://mlsanity.com/paper/2301.06277/) (2023)
- [Audio-Visual Speech Enhancement In Complex Scenarios With Separation And Dereverberation Joint Modeling](https://mlsanity.com/paper/2510.26825/) (2025)

## Abstract (arXiv)

Speaker extraction aims to extract the target speech signal from a multi-talker environment given a target speaker's reference speech. We recently proposed a time-domain solution, SpEx, that avoids the phase estimation in frequency-domain approaches. Unfortunately, SpEx is not fully a time-domain solution since it performs time-domain speech encoding for speaker extraction, while taking frequency-domain speaker embedding as the reference. The size of the analysis window for time-domain and the size for frequency-domain input are also different. Such mismatch has an adverse effect on the system performance. To eliminate such mismatch, we propose a complete time-domain speaker extraction solution, that is called SpEx+. Specifically, we tie the weights of two identical speech encoder networks, one for the encoder-extractor-decoder pipeline, another as part of the speaker encoder. Experiments show that the SpEx+ achieves 0.8dB and 2.1dB SDR improvement over the state-of-the-art SpEx baseline, under different and same gender conditions on WSJ0-2mix-extr database respectively.