mlsanityData to 14 Oct 2026

Phir Hera Fairy: An English Fairytaler is a Strong Faker of Fluent Speech in Low-Resource Indian Languages

arXiv 2505.20693 ↗First posted 27 May 2025cs.CL cs.LG cs.SDcited by 2

Not on the map. Why?

Abstract (arXiv)

What happens when an English Fairytaler is fine-tuned on Indian languages? We evaluate how the English F5-TTS model adapts to 11 Indian languages, measuring polyglot fluency, voice-cloning, style-cloning, and code-mixing. We compare: (i) training from scratch, (ii) fine-tuning English F5 on Indian data, and (iii) fine-tuning on both Indian and English data to prevent forgetting. Fine-tuning with only Indian data proves most effective and the resultant IN-F5 is a near-human polyglot; that enables speakers of one language (e.g., Odia) to fluently speak in another (e.g., Hindi). Our results show English pretraining aids low-resource TTS in reaching human parity. To aid progress in other low-resource languages, we study data-constrained setups and arrive at a compute optimal strategy. Finally, we show IN-F5 can synthesize unseen languages like Bhojpuri and Tulu using a human-in-the-loop approach for zero-resource TTS via synthetic data generation.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 21 of its 24 references that are in the map. First posted on arXiv 27 May 2025.

Builds on

21 of 24 references.

References · 21

  1. 2021WavLM: Large-Scale Self-Supervised Pre-Training for Full Stack Speech Processingcited by 1,657
  2. 2023Voicebox: Text-Guided Multilingual Universal Speech Generation at Scalecited by 379
  3. 2024F5-TTS: A Fairytaler that Fakes Fluent and Faithful Speech with Flow Matchingcited by 361
  4. 2024Seed-TTS: A Family of High-Quality Versatile Speech Generation Modelscited by 300
  5. 2024CosyVoice 2: Scalable Streaming Speech Synthesis with Large Language Modelscited by 288
  6. 2021YourTTS: Towards Zero-Shot Multi-Speaker TTS and Zero-Shot Voice Conversion for everyonecited by 278
  7. 2023NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizerscited by 235
  8. 2024NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Modelscited by 226
  9. 2024Emilia: An Extensive, Multilingual, and Diverse Speech Dataset for Large-Scale Speech Generationcited by 204
  10. 2024MaskGCT: Zero-Shot Text-to-Speech with Masked Generative Codec Transformercited by 161
  11. 2022NaturalSpeech: End-to-End Text to Speech Synthesis with Human-Level Qualitycited by 152
  12. 2024VoiceCraft: Zero-Shot Speech Editing and Text-to-Speech in the Wildcited by 120
9 more
  1. 2023SoundStorm: Efficient Parallel Audio Generationcited by 114
  2. 2024IndicVoices: Towards building an Inclusive Multilingual Speech Dataset for Indian Languagescited by 41
  3. 2022Towards Building Text-To-Speech Systems for the Next Billion Userscited by 27
  4. 2024IndicVoices-R: Unlocking a Massive Multilingual Multi-speaker Speech Corpus for Scaling Indian TTScited by 16
  5. 2024Rasa: Building Expressive Speech Synthesis Systems for Indian Languages in Low-resource Settingscited by 7
  6. 2024Rethinking MUSHRA: Addressing Modern Challenges in Text-to-Speech Evaluationcited by 7
  7. 2022Towards Developing State-of-the-Art TTS Synthesisers for 13 Indian Languages with Signal Processing aided Alignmentscited by 4
  8. 2024A Tulu Resource for Machine Translationcited by 4
  9. 2024Enhancing Out-of-Vocabulary Performance of Indian TTS Systems for Practical Applications through Low-Effort Data Strategiescited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Preferences of a Voice-First Nation: Large-Scale Pairwise Evaluation and Preference Analysis for TTS in Indian Languagescited by 1
  2. 2026CrossAccent-TTS: Cross-Lingual Accent-Intensity Controllable Text-to-Speech via Disentangled Speaker and Accent Representationscited by 1