# FlowTSE: Target Speaker Extraction with Flow Matching

arXiv: https://arxiv.org/abs/2505.14465 · HTML: https://mlsanity.com/paper/2505.14465/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Placed in Audio, speech and music, one of 14 directions in Diffusion models.
- Builds on 27 of its 35 references that are in the map.
- First posted on arXiv 20 May 2025.

## Placed in

- Diffusion models / [Audio, speech and music](https://mlsanity.com/diffusion/audio-speech-and-music/)

## Builds on (most cited first, 27 of 27)

- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Scalable Diffusion Models with Transformers](https://mlsanity.com/paper/2212.09748/) (2022)
- [Flow Matching for Generative Modeling](https://mlsanity.com/paper/2210.02747/) (2022)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matching](https://mlsanity.com/paper/2410.06885/) (2024)
- [Supervised Speech Separation Based on Deep Learning: An Overview](https://mlsanity.com/paper/1708.07524/) (2017)
- [Speech Enhancement and Dereverberation with Diffusion-based Generative Models](https://mlsanity.com/paper/2208.05830/) (2022)
- [DNSMOS P.835: A Non-Intrusive Perceptual Objective Speech Quality Metric to Evaluate Noise Suppressors](https://mlsanity.com/paper/2110.01763/) (2021)
- [WHAM!: Extending Speech Separation to Noisy Environments](https://mlsanity.com/paper/1907.01160/) (2019)
- [LibriMix: An Open-Source Dataset for Generalizable Speech Separation](https://mlsanity.com/paper/2005.11262/) (2020)
- [Vocos: Closing the gap between time-domain and Fourier-based neural vocoders for high-quality audio synthesis](https://mlsanity.com/paper/2306.00814/) (2023)
- [DCCRN: Deep Complex Convolution Recurrent Network for Phase-Aware Speech Enhancement](https://mlsanity.com/paper/2008.00264/) (2020)
- [Conditional Diffusion Probabilistic Model for Speech Enhancement](https://mlsanity.com/paper/2202.05256/) (2022)
- [StoRM: A Diffusion-based Stochastic Regeneration Model for Speech Enhancement and Dereverberation](https://mlsanity.com/paper/2212.11851/) (2022)
- [Diffusion-based Generative Speech Source Separation](https://mlsanity.com/paper/2210.17327/) (2022)
- [SpEx+: A Complete Time Domain Speaker Extraction Network](https://mlsanity.com/paper/2005.04686/) (2020)
- [Analysing Diffusion-based Generative Approaches versus Discriminative Approaches for Speech Restoration](https://mlsanity.com/paper/2211.02397/) (2022)
- [End-to-End Waveform Utterance Enhancement for Direct Evaluation Metrics Optimization by Fully Convolutional Neural Networks](https://mlsanity.com/paper/1709.03658/) (2017)
- [Target Speech Extraction with Conditional Diffusion Model](https://mlsanity.com/paper/2308.03987/) (2023)
- [TSELM: Target Speaker Extraction using Discrete Tokens and Language Models](https://mlsanity.com/paper/2409.07841/) (2024)
- [Target Speaker ASR with Whisper](https://mlsanity.com/paper/2409.09543/) (2024)
- [DDTSE: Discriminative Diffusion Model for Target Speech Extraction](https://mlsanity.com/paper/2309.13874/) (2023)
- [Diffusion-based speech enhancement with a weighted generative-supervised learning loss](https://mlsanity.com/paper/2309.10457/) (2023)
- [DPCCN: Densely-Connected Pyramid Complex Convolutional Network for Robust Speech Separation And Extraction](https://mlsanity.com/paper/2112.13520/) (2021)
- [Enhancing Intelligibility for Generative Target Speech Extraction via Joint Optimization with Target Speaker ASR](https://mlsanity.com/paper/2501.14477/) (2025)
- [Personalized speech enhancement combining band-split RNN and speaker attentive module](https://mlsanity.com/paper/2302.09953/) (2023)

## Built on by (most cited first, 6 of 6)

- [MeanFlow-TSE: One-Step Generative Target Speaker Extraction with Mean Flow](https://mlsanity.com/paper/2512.18572/) (2025)
- [Adaptive Deterministic Flow Matching for Target Speaker Extraction](https://mlsanity.com/paper/2510.16995/) (2025)
- [UmbraTTS: Adapting Text-to-Speech to Environmental Contexts with Flow Matching](https://mlsanity.com/paper/2506.09874/) (2025)
- [AlphaFlowTSE: One-Step Generative Target Speaker Extraction via Conditional AlphaFlow](https://mlsanity.com/paper/2603.10701/) (2026)
- [Mask2Flow-TSE: Two-Stage Target Speaker Extraction with Masking and Flow Matching](https://mlsanity.com/paper/2603.12837/) (2026)
- [Shared Representation Learning for Reference-Guided Targeted Sound Detection](https://mlsanity.com/paper/2603.17025/) (2026)

## Abstract (arXiv)

Target speaker extraction (TSE) aims to isolate a specific speaker's speech from a mixture using speaker enrollment as a reference. While most existing approaches are discriminative, recent generative methods for TSE achieve strong results. However, generative methods for TSE remain underexplored, with most existing approaches relying on complex pipelines and pretrained components, leading to computational overhead. In this work, we present FlowTSE, a simple yet effective TSE approach based on conditional flow matching. Our model receives an enrollment audio sample and a mixed speech signal, both represented as mel-spectrograms, with the objective of extracting the target speaker's clean speech. Furthermore, for tasks where phase reconstruction is crucial, we propose a novel vocoder conditioned on the complex STFT of the mixed signal, enabling improved phase estimation. Experimental results on standard TSE benchmarks show that FlowTSE matches or outperforms strong baselines.