# Transfer learning of language-independent end-to-end ASR with language model fusion

arXiv: https://arxiv.org/abs/1811.02134 · HTML: https://mlsanity.com/paper/1811.02134/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Nov 2018.

## Built on by (most cited first, 10 of 10)

- [XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale](https://mlsanity.com/paper/2111.09296/) (2021)
- [Recent Advances in End-to-End Automatic Speech Recognition](https://mlsanity.com/paper/2111.01690/) (2021)
- [XTREME-S: Evaluating Cross-lingual Speech Representations](https://mlsanity.com/paper/2203.10752/) (2022)
- [Leveraging Modality-specific Representations for Audio-visual Speech Recognition via Reinforcement Learning](https://mlsanity.com/paper/2212.05301/) (2022)
- [Massively Multilingual Shallow Fusion with Large Language Models](https://mlsanity.com/paper/2302.08917/) (2023)
- [Discovering Phonetic Inventories with Crosslingual Automatic Speech Recognition](https://mlsanity.com/paper/2201.11207/) (2022)
- [Exploring Machine Speech Chain for Domain Adaptation and Few-Shot Speaker Adaptation](https://mlsanity.com/paper/2104.03815/) (2021)
- [Overcoming Domain Mismatch in Low Resource Sequence-to-Sequence ASR Models using Hybrid Generated Pseudotranscripts](https://mlsanity.com/paper/2106.07716/) (2021)
- [A Deliberation-based Joint Acoustic and Text Decoder](https://mlsanity.com/paper/2303.15293/) (2023)
- [A Comprehensive Study of the Current State-of-the-Art in Nepali Automatic Speech Recognition Systems](https://mlsanity.com/paper/2402.03050/) (2024)

## Abstract (arXiv)

This work explores better adaptation methods to low-resource languages using an external language model (LM) under the framework of transfer learning. We first build a language-independent ASR system in a unified sequence-to-sequence (S2S) architecture with a shared vocabulary among all languages. During adaptation, we perform LM fusion transfer, where an external LM is integrated into the decoder network of the attention-based S2S model in the whole adaptation stage, to effectively incorporate linguistic context of the target language. We also investigate various seed models for transfer learning. Experimental evaluations using the IARPA BABEL data set show that LM fusion transfer improves performances on all target five languages compared with simple transfer learning when the external text data is available. Our final system drastically reduces the performance gap from the hybrid systems.