mlsanityData to 7 Oct 2026

Target Speech Extraction with Conditional Diffusion Model

arXiv 2308.03987 ↗First posted 8 Aug 2023eess.AS cs.SDcited by 15

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

Diffusion model-based speech enhancement has received increased attention since it can generate very natural enhanced signals and generalizes well to unseen conditions. Diffusion models have been explored for several sub-tasks of speech enhancement, such as speech denoising, dereverberation, and source separation. In this paper, we investigate their use for target speech extraction (TSE), which consists of estimating the clean speech signal of a target speaker in a mixture of multi-talkers. TSE is realized by conditioning the extraction process on a clue identifying the target speaker. We show we can realize TSE using a conditional diffusion model conditioned on the clue. Besides, we introduce ensemble inference to reduce potential extraction errors caused by the diffusion process. In experiments on Libri2mix corpus, we show that the proposed diffusion model-based TSE combined with ensemble inference outperforms a comparable TSE system trained discriminatively.

Key facts. Cited by 15 papers in the map (data through 7 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 24 of its 31 references that are in the map. First posted on arXiv 8 Aug 2023.

Builds on

24 of 31 references.

Closest ancestors · 6

Same direction, newest first

  1. 2022StoRM: A Diffusion-based Stochastic Regeneration Model for Speech Enhancement and Dereverberation22 Dec
  2. 2022Diffusion-based Generative Speech Source Separation31 Oct
  3. 2022Speech Enhancement and Dereverberation with Diffusion-based Generative Models11 Aug
  4. 2022Speech Enhancement with Score-Based Generative Models in the Complex STFT Domain31 Mar
  5. 2022Conditional Diffusion Probabilistic Model for Speech Enhancement10 Feb
  6. 2021Restoring degraded speech via a modified diffusion model22 Apr

Other references · 18

  1. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  2. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  3. 2022Classifier-Free Diffusion Guidancecited by 5,717
  4. 2021CSDI: Conditional Score-based Diffusion Models for Probabilistic Time Series Imputationcited by 577
  5. 2018SDR - half-baked or well done?cited by 420
  6. 2018Audio-Visual Scene Analysis with Self-Supervised Multisensory Featurescited by 335
  7. 2017Supervised Speech Separation Based on Deep Learning: An Overviewcited by 213
  8. 2021Conditional Image Generation with Score-Based Diffusion Modelscited by 195
  9. 2017SEGAN: Speech Enhancement Generative Adversarial Networkcited by 165
  10. 2020LibriMix: An Open-Source Dataset for Generalizable Speech Separationcited by 154
  11. 2018The Conversation: Deep Audio-Visual Speech Enhancementcited by 107
  12. 2022Universal Speech Enhancement with Score-based Diffusioncited by 75
6 more
  1. 2018VoiceFilter: Targeted Voice Separation by Speaker-Conditioned Spectrogram Maskingcited by 72
  2. 2017Statistical Speech Enhancement Based on Probabilistic Integration of Variational Autoencoder and Non-Negative Matrix Factorizationcited by 33
  3. 2020SpEx+: A Complete Time Domain Speaker Extraction Networkcited by 33
  4. 2023Neural Target Speech Extraction: An Overviewcited by 33
  5. 2020Improving speaker discrimination of target speech extraction with time-domain SpeakerBeamcited by 20
  6. 2019The sound of my voice: speaker representation loss for target voice separationcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 3; 2025: 7; 2026: 3.

15 papers.

Sort:

By area:

  1. 2023DPM-TSE: A Diffusion Probabilistic Model for Target Sound Extractioncited by 12
  2. 2024SoloAudio: Target Sound Extraction with Language-oriented Audio Diffusion Transformercited by 11
  3. 2024Noise-robust Speech Separation with Fast Generative Correctioncited by 9
  4. 2025SoloSpeech: Enhancing Intelligibility and Quality in Target Speech Extraction through a Cascaded Generative Pipelinecited by 9
  5. 2023DDTSE: Discriminative Diffusion Model for Target Speech Extractioncited by 8
  6. 2025FlowTSE: Target Speaker Extraction with Flow Matchingcited by 6
  7. 2025LauraTSE: Target Speaker Extraction using Auto-Regressive Decoder-Only Language Modelscited by 5
  8. 2025MeanFlow-TSE: One-Step Generative Target Speaker Extraction with Mean Flowcited by 4
  9. 2025Adaptive Deterministic Flow Matching for Target Speaker Extractioncited by 2
  10. 2024Flow-TSVAD: Target-Speaker Voice Activity Detection via Latent Flow Matchingcited by 1
  11. 2025Audio Generation Through Score-Based Generative Modeling: Design Principles and Implementationcited by 1
  12. 2026StarTSE: Towards Streaming Target Speaker Extraction via Chunk-wise Interleaved Splicing of Autoregressive Language Modelcited by 1
3 more
  1. 2025Combining Deterministic Enhanced Conditions with Dual-Streaming Encoding for Diffusion-Based Speech Enhancementcited by 0
  2. 2026AlphaFlowTSE: One-Step Generative Target Speaker Extraction via Conditional AlphaFlowcited by 0
  3. 2026Mask2Flow-TSE: Two-Stage Target Speaker Extraction with Masking and Flow Matchingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2021A Study on Speech Enhancement Based on Diffusion Probabilistic Model
  2. 2022TransFusion: Transcribing Speech with Multinomial Diffusion
  3. 2023Single and Few-step Diffusion for Generative Speech Enhancement
  4. 2023NADiffuSE: Noise-aware Diffusion-based Model for Speech Enhancement
  5. 2026Absorbing Discrete Diffusion for Speech Enhancement
  6. 2025GDiffuSE: Diffusion-based speech enhancement with noise model guidance
  7. 2025ProSE: Diffusion Priors for Speech Enhancement
  8. 2023Unsupervised speech enhancement with diffusion-based generative models