# ctPuLSE: Close-Talk, and Pseudo-Label Based Far-Field, Speech Enhancement arXiv: https://arxiv.org/abs/2407.19485 ยท HTML: https://mlsanity.com/paper/2407.19485/ ## Key facts - Cited by 2 papers in the map (data through 19 Oct 2026). - Placed in Speech enhancement and noise suppression, one of 11 directions in Speech and audio language models. - Builds on 32 of its 56 references that are in the map. - First posted on arXiv 28 Jul 2024. ## Placed in - Speech and audio language models / [Speech enhancement and noise suppression](https://mlsanity.com/speech/speech-enhancement-and-noise-suppression/) ## Builds on (most cited first, 30 of 32) - [WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processing](https://mlsanity.com/paper/2110.13900/) (2021) - [SDR - half-baked or well done?](https://mlsanity.com/paper/1811.02508/) (2018) - [DNSMOS: A Non-Intrusive Perceptual Objective Speech Quality metric to evaluate Noise Suppressors](https://mlsanity.com/paper/2010.15258/) (2020) - [Supervised Speech Separation Based on Deep Learning: An Overview](https://mlsanity.com/paper/1708.07524/) (2017) - [CHiME-6 Challenge:Tackling Multispeaker Speech Recognition for Unsegmented Recordings](https://mlsanity.com/paper/2004.09249/) (2020) - [M2MeT: The ICASSP 2022 Multi-Channel Multi-Party Meeting Transcription Challenge](https://mlsanity.com/paper/2110.07393/) (2021) - [TF-GridNet: Integrating Full- and Sub-Band Modeling for Speech Separation](https://mlsanity.com/paper/2211.12433/) (2022) - [The CHiME-7 DASR Challenge: Distant Meeting Transcription with Multiple Devices in Diverse Scenarios](https://mlsanity.com/paper/2306.13734/) (2023) - [Far-Field Automatic Speech Recognition](https://mlsanity.com/paper/2009.09395/) (2020) - [End-to-End Integration of Speech Recognition, Speech Enhancement, and Self-Supervised Learning Representation](https://mlsanity.com/paper/2204.00540/) (2022) - [RemixIT: Continual self-training of speech enhancement models via bootstrapped remixing](https://mlsanity.com/paper/2202.08862/) (2022) - [SpatialNet: Extensively Learning Spatial Information for Multichannel Joint Speech Separation, Denoising and Dereverberation](https://mlsanity.com/paper/2307.16516/) (2023) - [Multi-microphone Complex Spectral Mapping for Utterance-wise and Continuous Speech Separation](https://mlsanity.com/paper/2010.01703/) (2020) - [On The Compensation Between Magnitude and Phase in Speech Separation](https://mlsanity.com/paper/2108.05470/) (2021) - [Toward Universal Speech Enhancement for Diverse Input Conditions](https://mlsanity.com/paper/2309.17384/) (2023) - [AudioScopeV2: Audio-Visual Attention Architectures for Calibrated Open-Domain On-Screen Sound Separation](https://mlsanity.com/paper/2207.10141/) (2022) - [Multi-Microphone Complex Spectral Mapping for Speech Dereverberation](https://mlsanity.com/paper/2003.01861/) (2020) - [Convolutive Prediction for Monaural Speech Dereverberation and Noisy-Reverberant Speaker Separation](https://mlsanity.com/paper/2108.07376/) (2021) - [Towards Low-distortion Multi-channel Speech Enhancement: The ESPNet-SE Submission to The L3DAS22 Challenge](https://mlsanity.com/paper/2202.12298/) (2022) - [On Cross-Corpus Generalization of Deep Learning Based Speech Enhancement](https://mlsanity.com/paper/2002.04027/) (2020) - [Closing the Gap Between Time-Domain Multi-Channel Speech Enhancement on Real and Simulation Conditions](https://mlsanity.com/paper/2110.14139/) (2021) - [End-to-End Integration of Speech Recognition, Dereverberation, Beamforming, and Self-Supervised Learning Representation](https://mlsanity.com/paper/2210.10742/) (2022) - [The CHiME-7 UDASE task: Unsupervised domain adaptation for conversational speech enhancement](https://mlsanity.com/paper/2307.03533/) (2023) - [Convolutive Transfer Function Invariant SDR training criteria for Multi-Channel Reverberant Speech Separation](https://mlsanity.com/paper/2011.15003/) (2020) - [UNSSOR: Unsupervised Neural Speech Separation by Leveraging Over-determined Training Mixtures](https://mlsanity.com/paper/2305.20054/) (2023) - [The Multimodal Information Based Speech Processing (MISP) 2023 Challenge: Audio-Visual Target Speaker Extraction](https://mlsanity.com/paper/2309.08348/) (2023) - [Improving Design of Input Condition Invariant Speech Enhancement](https://mlsanity.com/paper/2401.14271/) (2024) - [Speaker Reinforcement Using Target Source Extraction for Robust Automatic Speech Recognition](https://mlsanity.com/paper/2205.04433/) (2022) - [Overview Of The 2023 Icassp Sp Clarity Challenge: Speech Enhancement For Hearing Aids](https://mlsanity.com/paper/2311.14490/) (2023) - [SuperM2M: Supervised and Mixture-to-Mixture Co-Learning for Speech Enhancement and Noise-Robust ASR](https://mlsanity.com/paper/2403.10271/) (2024) ## Built on by (most cited first, 2 of 2) - [FNSE-SBGAN: Far-field Speech Enhancement with Schrodinger Bridge and Generative Adversarial Networks](https://mlsanity.com/paper/2503.12936/) (2025) - [Cross-Talk Speech Reduction, by Separation, for Separation](https://mlsanity.com/paper/2605.19695/) (2026) ## Abstract (arXiv) The current dominant approach for neural speech enhancement is via purely-supervised deep learning on simulated pairs of far-field noisy-reverberant speech (i.e., mixtures) and clean speech. The trained models, however, often exhibit limited generalizability to real-recorded mixtures. To deal with this, this paper investigates training enhancement models directly on real mixtures. However, a major difficulty challenging this approach is that, since the clean speech of real mixtures is unavailable, there lacks a good supervision for real mixtures. In this context, assuming that a training set consisting of real-recorded pairs of close-talk and far-field mixtures is available, we propose to address this difficulty via close-talk speech enhancement, where an enhancement model is first trained on simulated mixtures to enhance real-recorded close-talk mixtures and the estimated close-talk speech can then be utilized as a supervision (i.e., pseudo-label) for training far-field speech enhancement models directly on the paired real-recorded far-field mixtures. We name the proposed system ctPuLSE. Evaluation results on the popular CHiME-4 dataset show that ctPuLSE can derive high-quality pseudo-labels and yield far-field speech enhancement models with strong generalizability to real data.