# NASTAR: Noise Adaptive Speech Enhancement with Target-Conditional Resampling

arXiv: https://arxiv.org/abs/2206.09058 · HTML: https://mlsanity.com/paper/2206.09058/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 43 references that are in the map.
- First posted on arXiv 18 Jun 2022.

## Builds on (most cited first, 21 of 21)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Momentum Contrast for Unsupervised Visual Representation Learning](https://mlsanity.com/paper/1911.05722/) (2019)
- [Domain-Adversarial Training of Neural Networks](https://mlsanity.com/paper/1505.07818/) (2015)
- [Prototypical Contrastive Learning of Unsupervised Representations](https://mlsanity.com/paper/2005.04966/) (2020)
- [SDR - half-baked or well done?](https://mlsanity.com/paper/1811.02508/) (2018)
- [Real Time Speech Enhancement in the Waveform Domain](https://mlsanity.com/paper/2006.12847/) (2020)
- [The INTERSPEECH 2020 Deep Noise Suppression Challenge: Datasets, Subjective Testing Framework, and Challenge Results](https://mlsanity.com/paper/2005.13981/) (2020)
- [Music Source Separation in the Waveform Domain](https://mlsanity.com/paper/1911.13254/) (2019)
- [TinyLSTMs: Efficient Neural Speech Enhancement for Hearing Aids](https://mlsanity.com/paper/2005.11138/) (2020)
- [Transfer Learning for Speech and Language Processing](https://mlsanity.com/paper/1511.06066/) (2015)
- [Self-supervised Text-independent Speaker Verification using Prototypical Momentum Contrastive Learning](https://mlsanity.com/paper/2012.07178/) (2020)
- [Learning Speaker Representations with Mutual Information](https://mlsanity.com/paper/1812.00271/) (2018)
- [Unsupervised Noise Adaptive Speech Enhancement by Discriminator-Constrained Optimal Transport](https://mlsanity.com/paper/2111.06316/) (2021)
- [Noise Adaptive Speech Enhancement using Domain Adversarial Training](https://mlsanity.com/paper/1807.07501/) (2018)
- [Deep Long Short-Term Memory Adaptive Beamforming Networks For Multichannel Robust Speech Recognition](https://mlsanity.com/paper/1711.08016/) (2017)
- [DNN-based Source Enhancement to Increase Objective Sound Quality Assessment Score](https://mlsanity.com/paper/1810.09137/) (2018)
- [Speech Enhancement In Multiple-Noise Conditions using Deep Neural Networks](https://mlsanity.com/paper/1605.02427/) (2016)
- [SERIL: Noise Adaptive Speech Enhancement using Regularization-based Incremental Learning](https://mlsanity.com/paper/2005.11760/) (2020)
- [Noise Tokens: Learning Neural Noise Templates for Environment-Aware Speech Enhancement](https://mlsanity.com/paper/2004.04001/) (2020)
- [Dynamic Noise Embedding: Noise Aware Training and Adaptation for Speech Enhancement](https://mlsanity.com/paper/2008.11920/) (2020)
- [Deep Xi as a Front-End for Robust Automatic Speech Recognition](https://mlsanity.com/paper/1906.07319/) (2019)

## Built on by (most cited first, 1 of 1)

- [LC4SV: A Denoising Framework Learning to Compensate for Unseen Speaker Verification Models](https://mlsanity.com/paper/2311.16604/) (2023)

## Abstract (arXiv)

For deep learning-based speech enhancement (SE) systems, the training-test acoustic mismatch can cause notable performance degradation. To address the mismatch issue, numerous noise adaptation strategies have been derived. In this paper, we propose a novel method, called noise adaptive speech enhancement with target-conditional resampling (NASTAR), which reduces mismatches with only one sample (one-shot) of noisy speech in the target environment. NASTAR uses a feedback mechanism to simulate adaptive training data via a noise extractor and a retrieval model. The noise extractor estimates the target noise from the noisy speech, called pseudo-noise. The noise retrieval model retrieves relevant noise samples from a pool of noise signals according to the noisy speech, called relevant-cohort. The pseudo-noise and the relevant-cohort set are jointly sampled and mixed with the source speech corpus to prepare simulated training data for noise adaptation. Experimental results show that NASTAR can effectively use one noisy speech sample to adapt an SE model to a target condition. Moreover, both the noise extractor and the noise retrieval model contribute to model adaptation. To our best knowledge, NASTAR is the first work to perform one-shot noise adaptation through noise extraction and retrieval.