# Effective Noise-aware Data Simulation for Domain-adaptive Speech Enhancement Leveraging Dynamic Stochastic Perturbation

arXiv: https://arxiv.org/abs/2409.01545 · HTML: https://mlsanity.com/paper/2409.01545/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 30 references that are in the map.
- First posted on arXiv 3 Sep 2024.

## Builds on (most cited first, 20 of 20)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [Adversarial Discriminative Domain Adaptation](https://mlsanity.com/paper/1702.05464/) (2017)
- [Contrastive Learning for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2007.15651/) (2020)
- [BEATs: Audio Pre-Training with Acoustic Tokenizers](https://mlsanity.com/paper/2212.09058/) (2022)
- [Unsupervised Domain Adaptation with Residual Transfer Networks](https://mlsanity.com/paper/1602.04433/) (2016)
- [A DIRT-T Approach to Unsupervised Domain Adaptation](https://mlsanity.com/paper/1802.08735/) (2018)
- [Real Time Speech Enhancement in the Waveform Domain](https://mlsanity.com/paper/2006.12847/) (2020)
- [MetricGAN: Generative Adversarial Networks based Black-box Metric Scores Optimization for Speech Enhancement](https://mlsanity.com/paper/1905.04874/) (2019)
- [StarGANv2-VC: A Diverse, Unsupervised, Non-parallel Framework for Natural-Sounding Voice Conversion](https://mlsanity.com/paper/2107.10394/) (2021)
- [CMGAN: Conformer-based Metric GAN for Speech Enhancement](https://mlsanity.com/paper/2203.15149/) (2022)
- [Transfer Learning for Speech and Language Processing](https://mlsanity.com/paper/1511.06066/) (2015)
- [Noise-robust Speech Recognition with 10 Minutes Unparalleled In-domain Data](https://mlsanity.com/paper/2203.15321/) (2022)
- [A Fully Convolutional Neural Network for Speech Enhancement](https://mlsanity.com/paper/1609.07132/) (2016)
- [Convolutional-Recurrent Neural Networks for Speech Enhancement](https://mlsanity.com/paper/1805.00579/) (2018)
- [Noise Adaptive Speech Enhancement using Domain Adversarial Training](https://mlsanity.com/paper/1807.07501/) (2018)
- [Efficient Transformer-based Speech Enhancement Using Long Frames and STFT Magnitudes](https://mlsanity.com/paper/2206.11703/) (2022)
- [Unsupervised Noise adaptation using Data Simulation](https://mlsanity.com/paper/2302.11981/) (2023)
- [Noise-aware Speech Enhancement using Diffusion Probabilistic Model](https://mlsanity.com/paper/2307.08029/) (2023)
- [SCP-GAN: Self-Correcting Discriminator Optimization for Training Consistency Preserving Metric GAN on Speech Enhancement Tasks](https://mlsanity.com/paper/2210.14474/) (2022)

## Built on by (most cited first, 1 of 1)

- [Universal Robust Speech Adaptation for Cross-Domain Speech Recognition and Enhancement](https://mlsanity.com/paper/2602.04307/) (2026)

## Abstract (arXiv)

Cross-domain speech enhancement (SE) is often faced with severe challenges due to the scarcity of noise and background information in an unseen target domain, leading to a mismatch between training and test conditions. This study puts forward a novel data simulation method to address this issue, leveraging noise-extractive techniques and generative adversarial networks (GANs) with only limited target noisy speech data. Notably, our method employs a noise encoder to extract noise embeddings from target-domain data. These embeddings aptly guide the generator to synthesize utterances acoustically fitted to the target domain while authentically preserving the phonetic content of the input clean speech. Furthermore, we introduce the notion of dynamic stochastic perturbation, which can inject controlled perturbations into the noise embeddings during inference, thereby enabling the model to generalize well to unseen noise conditions. Experiments on the VoiceBank-DEMAND benchmark dataset demonstrate that our domain-adaptive SE method outperforms an existing strong baseline based on data simulation.