# Improving noise robustness of automatic speech recognition via parallel data and teacher-student learning

arXiv: https://arxiv.org/abs/1901.02348 · HTML: https://mlsanity.com/paper/1901.02348/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jan 2019.

## Built on by (most cited first, 10 of 10)

- [Learning ASR pathways: A sparse multilingual ASR model](https://mlsanity.com/paper/2209.05735/) (2022)
- [CoDERT: Distilling Encoder Representations with Co-learning for Transducer-based Speech Recognition](https://mlsanity.com/paper/2106.07734/) (2021)
- [Multi-Modal Pre-Training for Automated Speech Recognition](https://mlsanity.com/paper/2110.09890/) (2021)
- [Collaborative Training of Acoustic Encoders for Speech Recognition](https://mlsanity.com/paper/2106.08960/) (2021)
- [Distribution Aware Metrics for Conditional Natural Language Generation](https://mlsanity.com/paper/2209.07518/) (2022)
- [TODM: Train Once Deploy Many Efficient Supernet-Based RNN-T Compression For On-device ASR Models](https://mlsanity.com/paper/2309.01947/) (2023)
- [Knowledge Distillation Leveraging Alternative Soft Targets from Non-Parallel Qualified Speech Data](https://mlsanity.com/paper/2112.08878/) (2021)
- [Multi-Stage Multi-Modal Pre-Training for Automatic Speech Recognition](https://mlsanity.com/paper/2403.19822/) (2024)
- [Sequence-level self-learning with multiple hypotheses](https://mlsanity.com/paper/2112.05826/) (2021)
- [End-to-end Jordanian dialect speech-to-text self-supervised learning framework](https://mlsanity.com/paper/2609.24410/) (2026)

## Abstract (arXiv)

For real-world speech recognition applications, noise robustness is still a challenge. In this work, we adopt the teacher-student (T/S) learning technique using a parallel clean and noisy corpus for improving automatic speech recognition (ASR) performance under multimedia noise. On top of that, we apply a logits selection method which only preserves the k highest values to prevent wrong emphasis of knowledge from the teacher and to reduce bandwidth needed for transferring data. We incorporate up to 8000 hours of untranscribed data for training and present our results on sequence trained models apart from cross entropy trained ones. The best sequence trained student model yields relative word error rate (WER) reductions of approximately 10.1%, 28.7% and 19.6% on our clean, simulated noisy and real test sets respectively comparing to a sequence trained teacher.