# Leveraging native language information for improved accented speech recognition

arXiv: https://arxiv.org/abs/1904.09038 · HTML: https://mlsanity.com/paper/1904.09038/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Apr 2019.

## Built on by (most cited first, 7 of 7)

- [Residual Adapters for Parameter-Efficient ASR Adaptation to Atypical and Accented Speech](https://mlsanity.com/paper/2109.06952/) (2021)
- [CommonAccent: Exploring Large Acoustic Pretrained Models for Accent Classification Based on Common Voice](https://mlsanity.com/paper/2305.18283/) (2023)
- [Accented Speech Recognition: A Survey](https://mlsanity.com/paper/2104.10747/) (2021)
- [Improving Mispronunciation Detection with Wav2vec2-based Momentum Pseudo-Labeling for Accentedness and Intelligibility Assessment](https://mlsanity.com/paper/2203.15937/) (2022)
- [Effects of Speaker Count, Duration, and Accent Diversity on Zero-Shot Accent Robustness in Low-Resource ASR](https://mlsanity.com/paper/2506.04364/) (2025)
- [Responsible Benchmarking of Fairness for Automatic Speech Recognition](https://mlsanity.com/paper/2605.10615/) (2026)
- [Clustering and Mining Accented Speech for Inclusive and Fair Speech Recognition](https://mlsanity.com/paper/2408.02582/) (2024)

## Abstract (arXiv)

Recognition of accented speech is a long-standing challenge for automatic speech recognition (ASR) systems, given the increasing worldwide population of bi-lingual speakers with English as their second language. If we consider foreign-accented speech as an interpolation of the native language (L1) and English (L2), using a model that can simultaneously address both languages would perform better at the acoustic level for accented speech. In this study, we explore how an end-to-end recurrent neural network (RNN) trained system with English and native languages (Spanish and Indian languages) could leverage data of native languages to improve performance for accented English speech. To this end, we examine pre-training with native languages, as well as multi-task learning (MTL) in which the main task is trained with native English and the secondary task is trained with Spanish or Indian Languages. We show that the proposed MTL model performs better than the pre-training approach and outperforms a baseline model trained simply with English data. We suggest a new setting for MTL in which the secondary task is trained with both English and the native language, using the same output set. This proposed scenario yields better performance with +11.95% and +17.55% character error rate gains over baseline for Hispanic and Indian accents, respectively.