mlsanityData to 14 Oct 2026

RASMALAI: Resources for Adaptive Speech Modeling in Indian Languages with Accents and Intonations

arXiv 2505.18609 ↗First posted 24 May 2025cs.CLcited by 4

Not on the map. Why?

Abstract (arXiv)

We introduce RASMALAI, a large-scale speech dataset with rich text descriptions, designed to advance controllable and expressive text-to-speech (TTS) synthesis for 23 Indian languages and English. It comprises 13,000 hours of speech and 24 million text-description annotations with fine-grained attributes like speaker identity, accent, emotion, style, and background conditions. Using RASMALAI, we develop IndicParlerTTS, the first open-source, text-description-guided TTS for Indian languages. Systematic evaluation demonstrates its ability to generate high-quality speech for named speakers, reliably follow text descriptions and accurately synthesize specified attributes. Additionally, it effectively transfers expressive characteristics both within and across languages. IndicParlerTTS consistently achieves strong performance across these evaluations, setting a new standard for controllable multilingual expressive speech synthesis in Indian languages.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 20 of its 22 references that are in the map. First posted on arXiv 24 May 2025.

Builds on

20 of 22 references.

References · 20

  1. 2024NaturalSpeech 3: Zero-Shot Speech Synthesis with Factorized Codec and Diffusion Modelscited by 226
  2. 2019pyannote.audio: neural building blocks for speaker diarizationcited by 181
  3. 2023IndicTrans2: Towards High-Quality and Accessible Machine Translation Models for all 22 Scheduled Indian Languagescited by 152
  4. 2023StyleTTS 2: Towards Human-Level Text-to-Speech through Style Diffusion and Adversarial Training with Large Speech Language Modelscited by 152
  5. 2024E2 TTS: Embarrassingly Easy Fully Non-Autoregressive Zero-Shot TTScited by 152
  6. 2023Audiobox: Unified Audio Generation with Natural Language Promptscited by 120
  7. 2024VoiceCraft: Zero-Shot Speech Editing and Text-to-Speech in the Wildcited by 120
  8. 2024Natural language guidance of high-fidelity text-to-speech with synthetic annotationscited by 67
  9. 2023PromptTTS 2: Describing and Generating Voices with Text Promptcited by 48
  10. 2023Mega-TTS 2: Boosting Prompting Mechanisms for Zero-Shot Speech Synthesiscited by 41
  11. 2024IndicVoices: Towards building an Inclusive Multilingual Speech Dataset for Indian Languagescited by 41
  12. 2024GLOBE: A High-quality English Corpus with Global Accents for Zero-shot Speaker Adaptive Text-to-Speechcited by 30
8 more
  1. 2023Stateful Conformer with Cache-based Inference for Streaming Automatic Speech Recognitioncited by 25
  2. 2022Brouhaha: multi-task training for voice activity detection, speech-to-noise ratio, and C50 room acoustics estimationcited by 23
  3. 2024An Empirical Comparison of Vocabulary Expansion and Initialization Approaches for Language Modelscited by 20
  4. 2022Speech Quality Assessment through MOS using Non-Matching Referencescited by 16
  5. 2023Cross-domain Neural Pitch and Periodicity Estimationcited by 16
  6. 2024IndicVoices-R: Unlocking a Massive Multilingual Multi-speaker Speech Corpus for Scaling Indian TTScited by 16
  7. 2024LibriTTS-P: A Corpus with Speaking Style and Speaker Identity Prompts for Text-to-Speech and Style Captioningcited by 15
  8. 2024Rasa: Building Expressive Speech Synthesis Systems for Indian Languages in Low-resource Settingscited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 4.

4 papers.

  1. 2026BhashaSutra: A Task-Centric Unified Survey of Indian NLP Datasets, Corpora, and Resourcescited by 1
  2. 2026Preferences of a Voice-First Nation: Large-Scale Pairwise Evaluation and Preference Analysis for TTS in Indian Languagescited by 1
  3. 2026Mind the Pause: Disfluency-Aware Objective Tuning for Multilingual Speech Correction with LLMscited by 1
  4. 2026Bolbosh: Script-Aware Flow Matching for Kashmiri Text-to-Speechcited by 0