# LRSpeech: Extremely Low-Resource Speech Synthesis and Recognition

arXiv: https://arxiv.org/abs/2008.03687 · HTML: https://mlsanity.com/paper/2008.03687/

## Key facts

- Cited by 43 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Aug 2020.

## Built on by (most cited first, 30 of 43)

- [A Survey on Neural Speech Synthesis](https://mlsanity.com/paper/2106.15561/) (2021)
- [MixSpeech: Data Augmentation for Low-resource Automatic Speech Recognition](https://mlsanity.com/paper/2102.12664/) (2021)
- [ASR2K: Speech Recognition for Around 2000 Languages without Audio](https://mlsanity.com/paper/2209.02842/) (2022)
- [Language-Agnostic Meta-Learning for Low-Resource Text-to-Speech with Articulatory Features](https://mlsanity.com/paper/2203.03191/) (2022)
- [How can NLP Help Revitalize Endangered Languages? A Case Study and Roadmap for the Cherokee Language](https://mlsanity.com/paper/2204.11909/) (2022)
- [EMOVIE: A Mandarin Emotion Speech Dataset with a Simple Emotional Text-to-Speech Model](https://mlsanity.com/paper/2106.09317/) (2021)
- [Integrated Speech and Gesture Synthesis](https://mlsanity.com/paper/2108.11436/) (2021)
- [Low-Resource Multilingual and Zero-Shot Multispeaker TTS](https://mlsanity.com/paper/2210.12223/) (2022)
- [Non-Autoregressive TTS with Explicit Duration Modelling for Low-Resource Highly Expressive Speech](https://mlsanity.com/paper/2106.12896/) (2021)
- [Unsupervised Text-to-Speech Synthesis by Unsupervised Automatic Speech Recognition](https://mlsanity.com/paper/2203.15796/) (2022)
- [Multilingual Byte2Speech Models for Scalable Low-resource Speech Synthesis](https://mlsanity.com/paper/2103.03541/) (2021)
- [OkwuGb'e: End-to-End Speech Recognition for Fon and Igbo](https://mlsanity.com/paper/2103.07762/) (2021)
- [Textless Unit-to-Unit training for Many-to-Many Multilingual Speech-to-Speech Translation](https://mlsanity.com/paper/2308.01831/) (2023)
- [Meta Learning Text-to-Speech Synthesis in over 7000 Languages](https://mlsanity.com/paper/2406.06403/) (2024)
- [Voice Filter: Few-shot text-to-speech speaker adaptation using voice conversion as a post-processing module](https://mlsanity.com/paper/2202.08164/) (2022)
- [Review of end-to-end speech synthesis technology based on deep learning](https://mlsanity.com/paper/2104.09995/) (2021)
- [Speech-to-SQL: Towards Speech-driven SQL Query Generation From Natural Language Question](https://mlsanity.com/paper/2201.01209/) (2022)
- [A Text-to-Speech Pipeline, Evaluation Methodology, and Initial Fine-Tuning Results for Child Speech Synthesis](https://mlsanity.com/paper/2203.11562/) (2022)
- [Zero-Shot Automatic Pronunciation Assessment](https://mlsanity.com/paper/2305.19563/) (2023)
- [Self-Supervised Audio-and-Text Pre-training with Extremely Low-Resource Parallel Data](https://mlsanity.com/paper/2204.04645/) (2022)
- [Unsupervised Data Selection for TTS: Using Arabic Broadcast News as a Case Study](https://mlsanity.com/paper/2301.09099/) (2023)
- [ParrotTTS: Text-to-Speech synthesis by exploiting self-supervised representations](https://mlsanity.com/paper/2303.01261/) (2023)
- [A multilingual training strategy for low resource Text to Speech](https://mlsanity.com/paper/2409.01217/) (2024)
- [Synthetic Voice Data for Automatic Speech Recognition in African Languages](https://mlsanity.com/paper/2507.17578/) (2025)
- [Towards Disentangled Speech Representations](https://mlsanity.com/paper/2208.13191/) (2022)
- [Few-Shot Cross-Lingual TTS Using Transferable Phoneme Embedding](https://mlsanity.com/paper/2206.15427/) (2022)
- [Towards High-Quality Neural TTS for Low-Resource Languages by Learning Compact Speech Representations](https://mlsanity.com/paper/2210.15131/) (2022)
- [ERSAM: Neural Architecture Search For Energy-Efficient and Real-Time Social Ambiance Measurement](https://mlsanity.com/paper/2303.10727/) (2023)
- [A Comparison of Semi-Supervised Learning Techniques for Streaming ASR at Scale](https://mlsanity.com/paper/2304.11053/) (2023)
- [Inclusion 2024 Global Multimedia Deepfake Detection Challenge: Towards Multi-dimensional Face Forgery Detection](https://mlsanity.com/paper/2412.20833/) (2024)

## Abstract (arXiv)

Speech synthesis (text to speech, TTS) and recognition (automatic speech recognition, ASR) are important speech tasks, and require a large amount of text and speech pairs for model training. However, there are more than 6,000 languages in the world and most languages are lack of speech training data, which poses significant challenges when building TTS and ASR systems for extremely low-resource languages. In this paper, we develop LRSpeech, a TTS and ASR system under the extremely low-resource setting, which can support rare languages with low data cost. LRSpeech consists of three key techniques: 1) pre-training on rich-resource languages and fine-tuning on low-resource languages; 2) dual transformation between TTS and ASR to iteratively boost the accuracy of each other; 3) knowledge distillation to customize the TTS model on a high-quality target-speaker voice and improve the ASR model on multiple voices. We conduct experiments on an experimental language (English) and a truly low-resource language (Lithuanian) to verify the effectiveness of LRSpeech. Experimental results show that LRSpeech 1) achieves high quality for TTS in terms of both intelligibility (more than 98% intelligibility rate) and naturalness (above 3.5 mean opinion score (MOS)) of the synthesized speech, which satisfy the requirements for industrial deployment, 2) achieves promising recognition accuracy for ASR, and 3) last but not least, uses extremely low-resource training data. We also conduct comprehensive analyses on LRSpeech with different amounts of data resources, and provide valuable insights and guidances for industrial deployment. We are currently deploying LRSpeech into a commercialized cloud speech service to support TTS on more rare languages.