# Reinforcement Learning Based Speech Enhancement for Robust Speech Recognition

arXiv: https://arxiv.org/abs/1811.04224 · HTML: https://mlsanity.com/paper/1811.04224/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Nov 2018.

## Built on by (most cited first, 10 of 10)

- [Automatic Speech Recognition using Advanced Deep Learning Approaches: A survey](https://mlsanity.com/paper/2403.01255/) (2024)
- [Reinforcement Learning for Emotional Text-to-Speech Synthesis with Improved Emotion Discriminability](https://mlsanity.com/paper/2104.01408/) (2021)
- [A Survey on Deep Reinforcement Learning for Audio-Based Applications](https://mlsanity.com/paper/2101.00240/) (2021)
- [Reinforcement Learning and Bandits for Speech and Language Processing: Tutorial, Review and Outlook](https://mlsanity.com/paper/2210.13623/) (2022)
- [A novel policy for pre-trained Deep Reinforcement Learning for Speech Emotion Recognition](https://mlsanity.com/paper/2101.00738/) (2021)
- [Improving Character Error Rate Is Not Equal to Having Clean Speech: Speech Enhancement for ASR Systems with Black-box Acoustic Models](https://mlsanity.com/paper/2110.05968/) (2021)
- [Denoising Induction Motor Sounds Using an Autoencoder](https://mlsanity.com/paper/2208.04462/) (2022)
- [AIIR-MIX: Multi-Agent Reinforcement Learning Meets Attention Individual Intrinsic Reward Mixing Network](https://mlsanity.com/paper/2302.09531/) (2023)
- [Autoregressive Speech Enhancement via Acoustic Tokens](https://mlsanity.com/paper/2507.12825/) (2025)
- [LLM-Guided Reinforcement Learning for Audio-Visual Speech Enhancement](https://mlsanity.com/paper/2603.13952/) (2026)

## Abstract (arXiv)

Conventional deep neural network (DNN)-based speech enhancement (SE) approaches aim to minimize the mean square error (MSE) between enhanced speech and clean reference. The MSE-optimized model may not directly improve the performance of an automatic speech recognition (ASR) system. If the target is to minimize the recognition error, the recognition results should be used to design the objective function for optimizing the SE model. However, the structure of an ASR system, which consists of multiple units, such as acoustic and language models, is usually complex and not differentiable. In this study, we proposed to adopt the reinforcement learning algorithm to optimize the SE model based on the recognition results. We evaluated the propsoed SE system on the Mandarin Chinese broadcast news corpus (MATBN). Experimental results demonstrate that the proposed method can effectively improve the ASR results with a notable 12.40% and 19.23% error rate reductions for signal to noise ratio at 0 dB and 5 dB conditions, respectively.