# Speech Technology for Everyone: Automatic Speech Recognition for Non-Native English with Transfer Learning

arXiv: https://arxiv.org/abs/2110.00678 · HTML: https://mlsanity.com/paper/2110.00678/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 38 references that are in the map.
- First posted on arXiv 1 Oct 2021.

## Builds on (most cited first, 19 of 19)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020)
- [Listen, Attend and Spell](https://mlsanity.com/paper/1508.01211/) (2015)
- [Libri-Light: A Benchmark for ASR with Limited or No Supervision](https://mlsanity.com/paper/1912.07875/) (2019)
- [Attention-Based Models for Speech Recognition](https://mlsanity.com/paper/1506.07503/) (2015)
- [vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations](https://mlsanity.com/paper/1910.05453/) (2019)
- [Mockingjay: Unsupervised Speech Representation Learning with Deep Bidirectional Transformer Encoders](https://mlsanity.com/paper/1910.12638/) (2019)
- [An Unsupervised Autoregressive Model for Speech Representation Learning](https://mlsanity.com/paper/1904.03240/) (2019)
- [Self-training and Pre-training are Complementary for Speech Recognition](https://mlsanity.com/paper/2010.11430/) (2020)
- [Transformer-based Acoustic Modeling for Hybrid Speech Recognition](https://mlsanity.com/paper/1910.09799/) (2019)
- [Deep Contextualized Acoustic Representations For Semi-Supervised Speech Recognition](https://mlsanity.com/paper/1912.01679/) (2019)
- [Domain Adversarial Training for Accented Speech Recognition](https://mlsanity.com/paper/1806.02786/) (2018)
- [The Accented English Speech Recognition Challenge 2020: Open Datasets, Tracks, Baselines, Results and Methods](https://mlsanity.com/paper/2102.10233/) (2021)
- [REDAT: Accent-Invariant Representation for End-to-End ASR by Domain Adversarial Training with Relabeling](https://mlsanity.com/paper/2012.07353/) (2020)
- [Best of Both Worlds: Robust Accented Speech Recognition with Adversarial Transfer Learning](https://mlsanity.com/paper/2103.05834/) (2021)
- [Non-native children speech recognition through transfer learning](https://mlsanity.com/paper/1809.09658/) (2018)
- [Independent language modeling architecture for end-to-end ASR](https://mlsanity.com/paper/1912.00863/) (2019)

## Built on by (most cited first, 10 of 10)

- [Adapting TTS models For New Speakers using Transfer Learning](https://mlsanity.com/paper/2110.05798/) (2021)
- [A Deep Dive into the Disparity of Word Error Rates Across Thousands of NPTEL MOOC Videos](https://mlsanity.com/paper/2307.10587/) (2023)
- [Improving the Inclusivity of Dutch Speech Recognition by Fine-tuning Whisper on the JASMIN-CGN Corpus](https://mlsanity.com/paper/2502.17284/) (2025)
- [Improving Automatic Speech Recognition for Non-Native English with Transfer Learning and Language Model Decoding](https://mlsanity.com/paper/2202.05209/) (2022)
- [Improving Children's Speech Recognition by Fine-tuning Self-supervised Adult Speech Representations](https://mlsanity.com/paper/2211.07769/) (2022)
- [INTapt: Information-Theoretic Adversarial Prompt Tuning for Enhanced Non-Native Speech Recognition](https://mlsanity.com/paper/2305.16371/) (2023)
- [LI-TTA: Language Informed Test-Time Adaptation for Automatic Speech Recognition](https://mlsanity.com/paper/2408.05769/) (2024)
- [Effects of Speaker Count, Duration, and Accent Diversity on Zero-Shot Accent Robustness in Low-Resource ASR](https://mlsanity.com/paper/2506.04364/) (2025)
- [Incorporating L2 Phonemes Using Articulatory Features for Robust Speech Recognition](https://mlsanity.com/paper/2306.02534/) (2023)
- [Revisiting Accent Classification with DistilHuBERT: Speaker-Disjoint Evaluation and Cross-Corpus Generalization](https://mlsanity.com/paper/2610.13440/) (2026)

## Abstract (arXiv)

To address the performance gap of English ASR models on L2 English speakers, we evaluate fine-tuning of pretrained wav2vec 2.0 models (Baevski et al., 2020; Xu et al., 2021) on L2-ARCTIC, a non-native English speech corpus (Zhao et al., 2018) under different training settings. We compare (a) models trained with a combination of diverse accents to ones trained with only specific accents and (b) results from different single-accent models. Our experiments demonstrate the promise of developing ASR models for non-native English speakers, even with small amounts of L2 training data and even without a language model. Our models also excel in the zero-shot setting where we train on multiple L2 datasets and test on a blind L2 test set.