ELAICHI: Enhancing Low-resource TTS by Addressing Infrequent and Low-frequency Character Bigrams
Not on the map. Why?
Abstract (arXiv)
Recent advancements in Text-to-Speech (TTS) technology have led to natural-sounding speech for English, primarily due to the availability of large-scale, high-quality web data. However, many other languages lack access to such resources, relying instead on limited studio-quality data. This scarcity results in synthesized speech that often suffers from intelligibility issues, particularly with low-frequency character bigrams. In this paper, we propose three solutions to address this challenge. First, we leverage high-quality data from linguistically or geographically related languages to improve TTS for the target language. Second, we utilize low-quality Automatic Speech Recognition (ASR) data recorded in non-studio environments, which is refined using denoising and speech enhancement models. Third, we apply knowledge distillation from large-scale models using synthetic data to generate more robust outputs. Our experiments with Hindi demonstrate significant reductions in intelligibility issues, as validated by human evaluators. We propose this methodology as a viable alternative for languages with limited access to high-quality data, enabling them to collectively benefit from shared resources.
Key facts. Cited by 2 papers in the map (data through 13 Oct 2026). Builds on 30 of its 38 references that are in the map. First posted on arXiv 23 Oct 2024.
Builds on
30 of 38 references.
References · 30
- 2017Attention Is All You Needcited by 60,106
- 2024The Llama 3 Herd of Modelscited by 14,486
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,004
- 2024Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phonecited by 1,908
- 2016WaveNet: A Generative Model for Raw Audiocited by 1,717
- 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,148
- 2021Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speechcited by 559
- 2020MLS: A Large-Scale Multilingual Dataset for Speech Researchcited by 468
- 2021GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audiocited by 393
- 2023NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizerscited by 235
- 2023Google USM: Scaling Automatic Speech Recognition Beyond 100 Languagescited by 229
- 2024NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Modelscited by 224
18 more
- 2021A Survey on Neural Speech Synthesiscited by 182
- 2024XTTS: a Massively Multilingual Zero-Shot Text-to-Speech Modelcited by 175
- 2023StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Modelscited by 152
- 2023IndicTrans2: Towards High-Quality and Accessible Machine Translation Models for all 22 Scheduled Indian Languagescited by 149
- 2022Towards Leaving No Indic Language Behind: Building Monolingual Corpora, Benchmark and Models for Indic Languagescited by 99
- 2023Distil-Whisper: Robust Knowledge Distillation via Large-Scale Pseudo Labellingcited by 87
- 2024IndicLLMSuite: A Blueprint for Creating Pre-training and Fine-Tuning Datasets for Indian Languagescited by 52
- 2021VoiceFixer: Toward General Speech Restoration with Neural Vocodercited by 50
- 2024IndicVoices: Towards building an Inclusive Multilingual Speech Dataset for Indian Languagescited by 39
- 2022Towards Building Text-To-Speech Systems for the Next Billion Userscited by 27
- 2022Brouhaha: multi-task training for voice activity detection, speech-to-noise ratio, and C50 room acoustics estimationcited by 21
- 2024IndicVoices-R: Unlocking a Massive Multilingual Multi-speaker Speech Corpus for Scaling Indian TTScited by 15
- 2020Google Crowdsourced Speech Corpora and Related Open-Source Resources for Low-Resource Languages and Dialects: An Overviewcited by 5
- 2022Distribution augmentation for low-resource expressive text-to-speechcited by 5
- 2022Nix-TTS: Lightweight and End-to-End Text-to-Speech via Module-wise Distillationcited by 4
- 2022Towards Developing State-of-the-Art TTS Synthesisers for 13 Indian Languages with Signal Processing aided Alignmentscited by 4
- 2024A multilingual training strategy for low resource Text to Speechcited by 3
- 2024Enhancing Out-of-Vocabulary Performance of Indian TTS Systems for Practical Applications through Low-Effort Data Strategiescited by 2
Built on by
Citing papers per year: 2025: 1; 2026: 1.
2 papers.