# Learning Cross-lingual Mappings for Data Augmentation to Improve Low-Resource Speech Recognition

arXiv: https://arxiv.org/abs/2306.08577 · HTML: https://mlsanity.com/paper/2306.08577/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 29 references that are in the map.
- First posted on arXiv 14 Jun 2023.

## Builds on (most cited first, 8 of 8)

- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates](https://mlsanity.com/paper/1804.10959/) (2018)
- [SpeechBrain: A General-Purpose Speech Toolkit](https://mlsanity.com/paper/2106.04624/) (2021)
- [Deep Speech: Scaling up end-to-end speech recognition](https://mlsanity.com/paper/1412.5567/) (2014)
- [Joint CTC-Attention based End-to-End Speech Recognition using Multi-task Learning](https://mlsanity.com/paper/1609.06773/) (2016)
- [Language-agnostic Multilingual Modeling](https://mlsanity.com/paper/2004.09571/) (2020)
- [Investigating the Impact of Cross-lingual Acoustic-Phonetic Similarities on Multilingual Speech Recognition](https://mlsanity.com/paper/2207.03390/) (2022)
- [Non-Linear Pairwise Language Mappings for Low-Resource Multilingual Acoustic Model Fusion](https://mlsanity.com/paper/2207.03391/) (2022)

## Built on by (most cited first, 1 of 1)

- [MUST: A Multilingual Student-Teacher Learning approach for low-resource speech recognition](https://mlsanity.com/paper/2310.18865/) (2023)

## Abstract (arXiv)

Exploiting cross-lingual resources is an effective way to compensate for data scarcity of low resource languages. Recently, a novel multilingual model fusion technique has been proposed where a model is trained to learn cross-lingual acoustic-phonetic similarities as a mapping function. However, handcrafted lexicons have been used to train hybrid DNN-HMM ASR systems. To remove this dependency, we extend the concept of learnable cross-lingual mappings for end-to-end speech recognition. Furthermore, mapping models are employed to transliterate the source languages to the target language without using parallel data. Finally, the source audio and its transliteration is used for data augmentation to retrain the target language ASR. The results show that any source language ASR model can be used for a low-resource target language recognition followed by proposed mapping model. Furthermore, data augmentation results in a relative gain up to 5% over baseline monolingual model.