# Leveraging supplementary text data to kick-start automatic speech recognition system development with limited transcriptions

arXiv: https://arxiv.org/abs/2302.04975 · HTML: https://mlsanity.com/paper/2302.04975/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 10 of its 22 references that are in the map.
- First posted on arXiv 9 Feb 2023.

## Builds on (most cited first, 10 of 10)

- [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale](https://mlsanity.com/paper/2111.09296/) (2021)
- [Libri-Light: A Benchmark for ASR with Limited or No Supervision](https://mlsanity.com/paper/1912.07875/) (2019)
- [Flashlight: Enabling Innovation in Tools for Machine Learning](https://mlsanity.com/paper/2201.12465/) (2022)
- [How can NLP Help Revitalize Endangered Languages? A Case Study and Roadmap for the Cherokee Language](https://mlsanity.com/paper/2204.11909/) (2022)
- [Adapting Monolingual Models: Data can be Scarce when Language Similarity is High](https://mlsanity.com/paper/2105.02855/) (2021)
- [Quantifying Language Variation Acoustically with Few Resources](https://mlsanity.com/paper/2205.02694/) (2022)
- [Transcribing Against Time](https://mlsanity.com/paper/1709.05227/) (2017)
- [Automated speech tools for helping communities process restricted-access corpora for language revival efforts](https://mlsanity.com/paper/2204.07272/) (2022)

## Built on by (most cited first, 4 of 4)

- [Making More of Little Data: Improving Low-Resource Automatic Speech Recognition Using Data Augmentation](https://mlsanity.com/paper/2305.10951/) (2023)
- [Multistage Fine-tuning Strategies for Automatic Speech Recognition in Low-resource Languages](https://mlsanity.com/paper/2411.04573/) (2024)
- [XLS-R Deep Learning Model for Multilingual ASR on Low- Resource Languages: Indonesian, Javanese, and Sundanese](https://mlsanity.com/paper/2401.06832/) (2024)
- [Exploring the Impact of Data Quantity on ASR in Extremely Low-resource Languages](https://mlsanity.com/paper/2409.08872/) (2024)

## Abstract (arXiv)

Recent research using pre-trained transformer models suggests that just 10 minutes of transcribed speech may be enough to fine-tune such a model for automatic speech recognition (ASR) -- at least if we can also leverage vast amounts of text data (803 million tokens). But is that much text data necessary? We study the use of different amounts of text data, both for creating a lexicon that constrains ASR decoding to possible words (e.g. *dogz vs. dogs), and for training larger language models that bias the system toward probable word sequences (e.g. too dogs vs. two dogs). We perform experiments using 10 minutes of transcribed speech from English (for replicating prior work) and two additional pairs of languages differing in the availability of supplemental text data: Gronings and Frisian (~7.5M token corpora available), and Besemah and Nasal (only small lexica available). For all languages, we found that using only a lexicon did not appreciably improve ASR performance. For Gronings and Frisian, we found that lexica and language models derived from 'novel-length' 80k token subcorpora reduced the word error rate (WER) to 39% on average. Our findings suggest that where a text corpus in the upper tens of thousands of tokens or more is available, fine-tuning a transformer model with just tens of minutes of transcribed speech holds some promise towards obtaining human-correctable transcriptions near the 30% WER rule-of-thumb.