# Noise-robust Speech Recognition with 10 Minutes Unparalleled In-domain Data

arXiv: https://arxiv.org/abs/2203.15321 · HTML: https://mlsanity.com/paper/2203.15321/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 20 references that are in the map.
- First posted on arXiv 29 Mar 2022.

## Builds on (most cited first, 13 of 13)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [Unsupervised Feature Learning via Non-Parametric Instance-level Discrimination](https://mlsanity.com/paper/1805.01978/) (2018)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [Contrastive Learning for Unpaired Image-to-Image Translation](https://mlsanity.com/paper/2007.15651/) (2020)
- [Generative Adversarial Networks: An Overview](https://mlsanity.com/paper/1710.07035/) (2017)
- [Listen, Attend and Spell](https://mlsanity.com/paper/1508.01211/) (2015)
- [Deep Speech: Scaling up end-to-end speech recognition](https://mlsanity.com/paper/1412.5567/) (2014)
- [A Comparative Study on Transformer vs RNN in Speech Applications](https://mlsanity.com/paper/1909.06317/) (2019)
- [State-of-the-art Speech Recognition With Sequence-to-Sequence Models](https://mlsanity.com/paper/1712.01769/) (2017)
- [StarGAN-VC: Non-parallel many-to-many voice conversion with star generative adversarial networks](https://mlsanity.com/paper/1806.02169/) (2018)
- [Multitask-Based Joint Learning Approach To Robust ASR For Radio Communication Speech](https://mlsanity.com/paper/2107.10701/) (2021)

## Built on by (most cited first, 30 of 30)

- [HyPoradise: An Open Baseline for Generative Speech Recognition with Large Language Models](https://mlsanity.com/paper/2309.15701/) (2023)
- [Leveraging Modality-specific Representations for Audio-visual Speech Recognition via Reinforcement Learning](https://mlsanity.com/paper/2212.05301/) (2022)
- [GenTranslate: Large Language Models are Generative Multilingual Speech and Machine Translators](https://mlsanity.com/paper/2402.06894/) (2024)
- [Speech Robust Bench: A Robustness Benchmark For Speech Recognition](https://mlsanity.com/paper/2403.07937/) (2024)
- [How "Real" is Your Real-Time Simultaneous Speech-to-Text Translation System?](https://mlsanity.com/paper/2412.18495/) (2024)
- [Personalized Adversarial Data Augmentation for Dysarthric and Elderly Speech Recognition](https://mlsanity.com/paper/2205.06445/) (2022)
- [Fairness-Aware Client Selection for Federated Learning](https://mlsanity.com/paper/2307.10738/) (2023)
- [Listen Again and Choose the Right Answer: A New Paradigm for Automatic Speech Recognition with Large Language Models](https://mlsanity.com/paper/2405.10025/) (2024)
- [Metric-oriented Speech Enhancement using Diffusion Probabilistic Model](https://mlsanity.com/paper/2302.11989/) (2023)
- [Unifying Speech Enhancement and Separation with Gradient Modulation for End-to-End Noise-Robust Speech Separation](https://mlsanity.com/paper/2302.11131/) (2023)
- [Gradient Remedy for Multi-Task Learning in End-to-End Noise-Robust Speech Recognition](https://mlsanity.com/paper/2302.11362/) (2023)
- [Wav2code: Restore Clean Speech Representations via Codebook Lookup for Noise-Robust ASR](https://mlsanity.com/paper/2304.04974/) (2023)
- [Failing Forward: Improving Generative Error Correction for ASR with Synthetic Data and Retrieval Augmentation](https://mlsanity.com/paper/2410.13198/) (2024)
- [Unsupervised Noise adaptation using Data Simulation](https://mlsanity.com/paper/2302.11981/) (2023)
- [Cross-Modal Global Interaction and Local Alignment for Audio-Visual Speech Recognition](https://mlsanity.com/paper/2305.09212/) (2023)
- [Self-critical Sequence Training for Automatic Speech Recognition](https://mlsanity.com/paper/2204.06260/) (2022)
- [Self-Taught Recognizer: Toward Unsupervised Adaptation for Speech Foundation Models](https://mlsanity.com/paper/2405.14161/) (2024)
- [Hearing Lips in Noise: Universal Viseme-Phoneme Mapping and Transfer for Robust Audio-Visual Speech Recognition](https://mlsanity.com/paper/2306.10563/) (2023)
- [Speech-text based multi-modal training with bidirectional attention for improved speech recognition](https://mlsanity.com/paper/2211.00325/) (2022)
- [MIR-GAN: Refining Frame-Level Modality-Invariant Representations with Adversarial Network for Audio-Visual Speech Recognition](https://mlsanity.com/paper/2306.10567/) (2023)
- [Channel-Aware Domain-Adaptive Generative Adversarial Network for Robust Speech Recognition](https://mlsanity.com/paper/2409.12386/) (2024)
- [Aligning Generative Speech Enhancement with Perceptual Feedback](https://mlsanity.com/paper/2507.09929/) (2025)
- [Seeking Next Layer Neurons' Attention for Error-Backpropagation-Like Training in a Multi-Agent Network Framework](https://mlsanity.com/paper/2310.09952/) (2023)
- [Fairness-Aware Job Scheduling for Multi-Job Federated Learning](https://mlsanity.com/paper/2401.02740/) (2024)
- [Effective Noise-aware Data Simulation for Domain-adaptive Speech Enhancement Leveraging Dynamic Stochastic Perturbation](https://mlsanity.com/paper/2409.01545/) (2024)
- [DAG-AFL:Directed Acyclic Graph-based Asynchronous Federated Learning](https://mlsanity.com/paper/2507.20571/) (2025)
- [AudioScene: Integrating Object-Event Audio into 3D Scenes](https://mlsanity.com/paper/2512.07845/) (2025)
- [Universal Robust Speech Adaptation for Cross-Domain Speech Recognition and Enhancement](https://mlsanity.com/paper/2602.04307/) (2026)
- [Cross-Modal Robustness Transfer (CMRT): Training Robust Speech Translation Models Using Adversarial Text](https://mlsanity.com/paper/2602.11933/) (2026)
- [Error-Aware TF-IDF Retrieval-Augmented Generation for ASR Error Correction](https://mlsanity.com/paper/2606.24915/) (2026)

## Abstract (arXiv)

Noise-robust speech recognition systems require large amounts of training data including noisy speech data and corresponding transcripts to achieve state-of-the-art performances in face of various practical environments. However, such plenty of in-domain data is not always available in the real-life world. In this paper, we propose a generative adversarial network to simulate noisy spectrum from the clean spectrum (Simu-GAN), where only 10 minutes of unparalleled in-domain noisy speech data is required as labels. Furthermore, we also propose a dual-path speech recognition system to improve the robustness of the system under noisy conditions. Experimental results show that the proposed speech recognition system achieves 7.3% absolute improvement with simulated noisy data by Simu-GAN over the best baseline in terms of word error rate (WER).