mlsanityData to 14 Oct 2026

Rasa: Building Expressive Speech Synthesis Systems for Indian Languages in Low-resource Settings

arXiv 2407.14056 ↗First posted 19 Jul 2024cs.CL cs.LG cs.SDcited by 7

Not on the map. Why?

Abstract (arXiv)

We release Rasa, the first multilingual expressive TTS dataset for any Indian language, which contains 10 hours of neutral speech and 1-3 hours of expressive speech for each of the 6 Ekman emotions covering 3 languages: Assamese, Bengali, & Tamil. Our ablation studies reveal that just 1 hour of neutral and 30 minutes of expressive data can yield a Fair system as indicated by MUSHRA scores. Increasing neutral data to 10 hours, with minimal expressive data, significantly enhances expressiveness. This offers a practical recipe for resource-constrained languages, prioritizing easily obtainable neutral data alongside smaller amounts of expressive data. We show the importance of syllabically balanced data and pooling emotions to enhance expressiveness. We also highlight challenges in generating specific emotions, e.g., fear and surprise.

Key facts. Cited by 7 papers in the map (data through 14 Oct 2026). Builds on 23 of its 40 references that are in the map. First posted on arXiv 19 Jul 2024.

Builds on

23 of 40 references.

References · 23

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesiscited by 1,154
  3. 2018Style Tokens: Unsupervised Style Modeling, Control and Transfer in End-to-End Speech Synthesiscited by 259
  4. 2023IndicTrans2: Towards High-Quality and Accessible Machine Translation Models for all 22 Scheduled Indian Languagescited by 152
  5. 2020FastPitch: Parallel Text-to-speech with Pitch Predictioncited by 151
  6. 2020Seen and Unseen emotional style transfer for voice conversion with a new emotional speech datasetcited by 121
  7. 2023EXPRESSO: A Benchmark and Analysis of Discrete Expressive Speech Resynthesiscited by 98
  8. 2024IndicLLMSuite: A Blueprint for Creating Pre-training and Fine-Tuning Datasets for Indian Languagescited by 52
  9. 2021One TTS Alignment To Rule Them Allcited by 47
  10. 2022EmoDiff: Intensity Controllable Emotional Text-to-Speech with Soft-Label Guidancecited by 37
  11. 2017Emotional End-to-End Neural Speech Synthesizercited by 30
  12. 2022MsEmoTTS: Multi-scale emotion transfer, prediction, and control for emotional speech synthesiscited by 29
11 more
  1. 2022Towards Building Text-To-Speech Systems for the Next Billion Userscited by 27
  2. 2020Low-resource expressive text-to-speech using data augmentationcited by 26
  3. 2021Towards Multi-Scale Style Control for Expressive Speech Synthesiscited by 18
  4. 2021EMOVIE: A Mandarin Emotion Speech Dataset with a Simple Emotional Text-to-Speech Modelcited by 18
  5. 2022Low-Resource Multilingual and Zero-Shot Multispeaker TTScited by 15
  6. 2020Att-HACK: An Expressive Speech Database with Social Attitudescited by 13
  7. 2021Multilingual Byte2Speech Models for Scalable Low-resource Speech Synthesiscited by 13
  8. 2021Adapting TTS models For New Speakers using Transfer Learningcited by 12
  9. 2020Generic Indic Text-to-speech Synthesisers with Rapid Adaptation in an End-to-end Frameworkcited by 9
  10. 2022Cross-Speaker Emotion Transfer for Low-Resource Text-to-Speech Using Non-Parallel Voice Conversion with Pitch-Shift Data Augmentationcited by 8
  11. 2023Diversity-based core-set selection for text-to-speech with linguistic and acoustic featurescited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 4.

7 papers.

  1. 2026OmniVoice: Towards Omnilingual Zero-Shot Text-to-Speech with Diffusion Language Modelscited by 25
  2. 2024Rethinking MUSHRA: Addressing Modern Challenges in Text-to-Speech Evaluationcited by 7
  3. 2025RASMALAI: Resources for Adaptive Speech Modeling in Indian Languages with Accents and Intonationscited by 4
  4. 2025Phir Hera Fairy: An English Fairytaler is a Strong Faker of Fluent Speech in Low-Resource Indian Languagescited by 2
  5. 2026BhashaSutra: A Task-Centric Unified Survey of Indian NLP Datasets, Corpora, and Resourcescited by 1
  6. 2026Preferences of a Voice-First Nation: Large-Scale Pairwise Evaluation and Preference Analysis for TTS in Indian Languagescited by 1
  7. 2026Bolbosh: Script-Aware Flow Matching for Kashmiri Text-to-Speechcited by 0