mlsanityData to 7 Oct 2026

VoiceFilter: Targeted Voice Separation by Speaker-Conditioned Spectrogram Masking

arXiv 1810.04826 ↗First posted 11 Oct 2018eess.AS cs.LG eess.SPcited by 72

Not on the map. Why?

Abstract (arXiv)

In this paper, we present a novel system that separates the voice of a target speaker from multi-speaker signals, by making use of a reference signal from the target speaker. We achieve this by training two separate neural networks: (1) A speaker recognition network that produces speaker-discriminative embeddings; (2) A spectrogram masking network that takes both noisy spectrogram and speaker embedding as input, and produces a mask. Our system significantly reduces the speech recognition WER on multi-speaker signals, with minimal WER degradation on single-speaker signals.

Key facts. Cited by 72 papers in the map (data through 7 Oct 2026). First posted on arXiv 11 Oct 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 18; 2022: 17; 2023: 18; 2024: 7; 2025: 5; 2026: 7.

72 papers.

Sort:

By area:

  1. 2023SpeechX: Neural Codec Language Model as a Versatile Speech Transformercited by 64
  2. 2022Separate What You Describe: Language-Queried Audio Source Separationcited by 45
  3. 2023Transformers in Speech Processing: A Surveycited by 38
  4. 2021MAAS: Multi-modal Assignation for Active Speaker Detectioncited by 35
  5. 2021How to Design a Three-Stage Architecture for Audio-Visual Active Speaker Detection in the Wildcited by 30
  6. 2021Looking into Your Speech: Learning Cross-modal Affinity for Audio-visual Speech Separationcited by 27
  7. 2022AudioScopeV2: Audio-Visual Attention Architectures for Calibrated Open-Domain On-Screen Sound Separationcited by 24
  8. 2022Real-Time Target Sound Extractioncited by 22
  9. 2024ESPnet-SPK: full pipeline speaker embedding toolkit with reproducible recipes, self-supervised front-ends, and off-the-shelf modelscited by 22
  10. 2021Personalized Speech Enhancement: New Models and Comprehensive Evaluationcited by 18
  11. 2022SpeechPainter: Text-conditioned Speech Inpaintingcited by 17
  12. 2023Target Speech Extraction with Conditional Diffusion Modelcited by 15
38 more
  1. 2022Heterogeneous Target Speech Separationcited by 13
  2. 2023DPM-TSE: A Diffusion Probabilistic Model for Target Sound Extractioncited by 12
  3. 2024Look Once to Hear: Target Speech Hearing with Noisy Examplescited by 11
  4. 2023ReZero: Region-customizable Sound Extractioncited by 10
  5. 2023FedTherapist: Mental Health Monitoring with User-Generated Linguistic Expressions on Smartphones via Federated Learningcited by 10
  6. 2025SoloSpeech: Enhancing Intelligibility and Quality in Target Speech Extraction through a Cascaded Generative Pipelinecited by 9
  7. 2021Personalized Keyphrase Detection using Speaker and Environment Informationcited by 8
  8. 2022Learning Lip-Based Audio-Visual Speaker Embeddings with AV-HuBERTcited by 8
  9. 2023DDTSE: Discriminative Diffusion Model for Target Speech Extractioncited by 8
  10. 2021Multi-Channel Multi-Speaker ASR Using 3D Spatial Featurecited by 7
  11. 2023X-SepFormer: End-to-end Speaker Extraction Network with Explicit Optimization on Speaker Confusioncited by 6
  12. 2023LocSelect: Target Speaker Localization with an Auditory Selective Hearing Mechanismcited by 6
  13. 2024Listen, Chat, and Remix: Text-Guided Soundscape Remixing for Enhanced Auditory Experiencecited by 6
  14. 2021Efficient Personalized Speech Enhancement through Self-Supervised Learningcited by 5
  15. 2021Personalized Speech Enhancement through Self-Supervised Data Augmentation and Purificationcited by 5
  16. 2021Improving On-Screen Sound Separation for Open-Domain Videos with Audio-Visual Self-Attentioncited by 5
  17. 2021Directed Speech Separation for Automatic Speech Recognition of Long Form Conversational Speechcited by 5
  18. 2022A Comparative Study on Speaker-attributed Automatic Speech Recognition in Multi-party Meetingscited by 5
  19. 2022Speaker Extraction with Co-Speech Gestures Cuecited by 5
  20. 2022Optimal Condition Training for Target Source Separationcited by 5
  21. 2023Self-Supervised Disentangled Representation Learning for Robust Target Speech Extractioncited by 5
  22. 2021Zero-Shot Personalized Speech Enhancement through Speaker-Informed Model Selectioncited by 4
  23. 2021Multi-user VoiceFilter-Lite via Attentive Speaker Embeddingcited by 4
  24. 2022Spatially Selective Deep Non-linear Filters for Speaker Extractioncited by 4
  25. 2023BASEN: Time-Domain Brain-Assisted Speech Enhancement Network with Convolutional Cross Attention in Multi-talker Conditionscited by 4
  26. 2023End-to-End Joint Target and Non-Target Speakers ASRcited by 4
  27. 2024Cross-attention Inspired Selective State Space Models for Target Sound Extractioncited by 4
  28. 2025UniFlow: Unifying Speech Front-End Tasks via Continuous Generative Modelingcited by 4
  29. 2025MeanFlow-TSE: One-Step Generative Target Speaker Extraction with Mean Flowcited by 4
  30. 2021WASE: Learning When to Attend for Speaker Extraction in Cocktail Party Environmentscited by 3
  31. 2022Quantitative Evidence on Overlooked Aspects of Enrollment Speaker Embeddings for Target Speaker Separationcited by 3
  32. 2022Deep neural network techniques for monaural speech enhancement: state of the art analysiscited by 3
  33. 2021Time-Domain Speech Extraction with Spatial Information and Multi Speaker Conditioning Mechanismcited by 2
  34. 2021Tune-In: Training Under Negative Environments with Interference for Attention Networks Simulating Cocktail Party Effectcited by 2
  35. 2021BeamTransformer: Microphone Array-based Overlapping Speech Detectioncited by 2
  36. 2023Convoifilter: A case study of doing cocktail party speech recognitioncited by 2
  37. 2026Cocktail-Talker: Multi-Speaker Dialog Modeling in Noisy Social Environments with Turn Action GRPOcited by 2
  38. 2021A Neural Text-to-Speech Model Utilizing Broadcast Data Mixed with Background Musiccited by 1

50 most cited of 72 All 72 (JSON)