# Effectiveness of Mining Audio and Text Pairs from Public Data for Improving ASR Systems for Low-Resource Languages arXiv: https://arxiv.org/abs/2208.12666 ยท HTML: https://mlsanity.com/paper/2208.12666/ ## Key facts - Cited by 16 papers in the map (data through 14 Oct 2026). - Builds on 14 of its 44 references that are in the map. - First posted on arXiv 26 Aug 2022. ## Builds on (most cited first, 14 of 14) - [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017) - [wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representations](https://mlsanity.com/paper/2006.11477/) (2020) - [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015) - [Conformer: Convolution-augmented Transformer for Speech Recognition](https://mlsanity.com/paper/2005.08100/) (2020) - [Common Voice: A Massively-Multilingual Speech Corpus](https://mlsanity.com/paper/1912.06670/) (2019) - [XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale](https://mlsanity.com/paper/2111.09296/) (2021) - [Recent Advances in End-to-End Automatic Speech Recognition](https://mlsanity.com/paper/2111.01690/) (2021) - [Deep context: end-to-end contextual speech recognition](https://mlsanity.com/paper/1808.02480/) (2018) - [Samanantar: The Largest Publicly Available Parallel Corpora Collection for 11 Indic Languages](https://mlsanity.com/paper/2104.05596/) (2021) - [An analysis of incorporating an external language model into a sequence-to-sequence model](https://mlsanity.com/paper/1712.01996/) (2017) - [Multilingual and code-switching ASR challenges for low resource Indian languages](https://mlsanity.com/paper/2104.00235/) (2021) - [Towards Building ASR Systems for the Next Billion Users](https://mlsanity.com/paper/2111.03945/) (2021) - [IndicSUPERB: A Speech Processing Universal Performance Benchmark for Indian languages](https://mlsanity.com/paper/2208.11761/) (2022) - [Neural Speech Recognizer: Acoustic-to-Word LSTM Model for Large Vocabulary Speech Recognition](https://mlsanity.com/paper/1610.09975/) (2016) ## Built on by (most cited first, 16 of 16) - [Vistaar: Diverse Benchmarks and Training Sets for Indian Language ASR](https://mlsanity.com/paper/2305.15386/) (2023) - [IndicVoices-R: Unlocking a Massive Multilingual Multi-speaker Speech Corpus for Scaling Indian TTS](https://mlsanity.com/paper/2409.05356/) (2024) - [OOD-Speech: A Large Bengali Speech Recognition Dataset for Out-of-Distribution Benchmarking](https://mlsanity.com/paper/2305.09688/) (2023) - [A Novel Self-training Approach for Low-resource Speech Recognition](https://mlsanity.com/paper/2308.05269/) (2023) - [Empowering Low-Resource Language ASR via Large-Scale Pseudo Labeling](https://mlsanity.com/paper/2408.14026/) (2024) - [Can we train ASR systems on Code-switch without real code-switch data? Case study for Singapore's languages](https://mlsanity.com/paper/2506.14177/) (2025) - [NIRANTAR: Continual Learning with New Languages and Domains on Real-world Speech Data](https://mlsanity.com/paper/2507.00534/) (2025) - [DuDe: Dual-Decoder Multilingual ASR for Indian Languages using Common Label Set](https://mlsanity.com/paper/2210.16739/) (2022) - [End-to-End Speech Translation for Low-Resource Languages Using Weakly Labeled Data](https://mlsanity.com/paper/2506.16251/) (2025) - [WorldSpeech: A Multilingual Speech Corpus from Around the World](https://mlsanity.com/paper/2605.09167/) (2026) - [Vividh-ASR: A Complexity-Tiered Benchmark and Optimization Dynamics for Robust Indic Speech Recognition](https://mlsanity.com/paper/2605.13087/) (2026) - [SCRIBE: Diagnostic Evaluation and Rich Transcription Models for Indic ASR](https://mlsanity.com/paper/2605.20712/) (2026) - [Breaking the Script Barrier: Enabling Automatic Alignment for PoS-based ASR Error Analysis in Non-Latin Scripts](https://mlsanity.com/paper/2605.28438/) (2026) - [Multilingual Multi-Speaker Unit Vocoders: A Systematic Analysis of Discrete Speech Representations](https://mlsanity.com/paper/2606.06740/) (2026) - [Vimarsha: Faithful ASR Evaluation for Indian Languages with Demographic Diversity, In-the-Wild Audio and Spelling Variations](https://mlsanity.com/paper/2609.24199/) (2026) - [BanglaKontho: Closing the Long-Form Gap in Bangla Text-to-Speech](https://mlsanity.com/paper/2609.29146/) (2026) ## Abstract (arXiv) End-to-end (E2E) models have become the default choice for state-of-the-art speech recognition systems. Such models are trained on large amounts of labelled data, which are often not available for low-resource languages. Techniques such as self-supervised learning and transfer learning hold promise, but have not yet been effective in training accurate models. On the other hand, collecting labelled datasets on a diverse set of domains and speakers is very expensive. In this work, we demonstrate an inexpensive and effective alternative to these approaches by ``mining'' text and audio pairs for Indian languages from public sources, specifically from the public archives of All India Radio. As a key component, we adapt the Needleman-Wunsch algorithm to align sentences with corresponding audio segments given a long audio and a PDF of its transcript, while being robust to errors due to OCR, extraneous text, and non-transcribed speech. We thus create Shrutilipi, a dataset which contains over 6,400 hours of labelled audio across 12 Indian languages totalling to 4.95M sentences. On average, Shrutilipi results in a 2.3x increase over publicly available labelled data. We establish the quality of Shrutilipi with 21 human evaluators across the 12 languages. We also establish the diversity of Shrutilipi in terms of represented regions, speakers, and mentioned named entities. Significantly, we show that adding Shrutilipi to the training set of Wav2Vec models leads to an average decrease in WER of 5.8% for 7 languages on the IndicSUPERB benchmark. For Hindi, which has the most benchmarks (7), the average WER falls from 18.8% to 13.5%. This improvement extends to efficient models: We show a 2.3% drop in WER for a Conformer model (10x smaller than Wav2Vec). Finally, we demonstrate the diversity of Shrutilipi by showing that the model trained with it is more robust to noisy input.