mlsanityData to 13 Oct 2026

Towards Building Large Scale Datasets and State-of-the-Art Automatic Speech Translation Systems for 14 Indian Languages

arXiv 2411.04699 ↗First posted 7 Nov 2024cs.CLcited by 4

Not on the map. Why?

Abstract (arXiv)

Speech translation for Indian languages remains a challenging task due to the scarcity of large-scale, publicly available datasets that capture the linguistic diversity and domain coverage essential for real-world applications. Existing datasets cover a fraction of Indian languages and lack the breadth needed to train robust models that generalize beyond curated benchmarks. To bridge this gap, we introduce BhasaAnuvaad, the largest speech translation dataset for Indian languages, spanning over 44 thousand hours of audio and 17 million aligned text segments across 14 Indian languages and English. Our dataset is built through a threefold methodology: (a) aggregating high-quality existing sources, (b) large-scale web crawling to ensure linguistic and domain diversity, and (c) creating synthetic data to model real-world speech disfluencies. Leveraging BhasaAnuvaad, we train IndicSeamless, a state-of-the-art speech translation model for Indian languages that performs better than existing models. Our experiments demonstrate improvements in the translation quality, setting a new standard for Indian language speech translation. We will release all the code, data and model weights in the open-source, with permissive licenses to promote accessibility and collaboration.

Key facts. Cited by 4 papers in the map (data through 13 Oct 2026). Builds on 29 of its 49 references that are in the map. First posted on arXiv 7 Nov 2024.

Builds on

29 of 49 references.

References · 29

  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,004
  2. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,793
  3. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784
  4. 2023Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Modelscited by 601
  5. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 594
  6. 2022FLEURS: Few-shot Learning Evaluation of Universal Representations of Speechcited by 459
  7. 2023xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detectioncited by 260
  8. 2023IndicTrans2: Towards High-Quality and Accessible Machine Translation Models for all 22 Scheduled Indian Languagescited by 149
  9. 2021Direct speech-to-speech translation with discrete unitscited by 148
  10. 2023On decoder-only architecture for speech-to-text and large language model integrationcited by 144
  11. 2017Sequence-to-Sequence Models Can Directly Translate Foreign Speechcited by 121
  12. 2022mSLAM: Massively multilingual joint pre-training for speech and textcited by 108
17 more
  1. 2023SONAR: Sentence-Level Multimodal and Language-Agnostic Representationscited by 108
  2. 2021The Multilingual TEDx Corpus for Speech Recognition and Translationcited by 101
  3. 2020Speech Translation and the End-to-End Promise: Taking Stock of Where We Arecited by 77
  4. 2023AudioChatLlama: Towards General-Purpose Speech Abilities for LLMscited by 59
  5. 2021Cascade versus Direct Speech Translation: Do the Differences Still Make a Difference?cited by 54
  6. 2024IndicVoices: Towards building an Inclusive Multilingual Speech Dataset for Indian Languagescited by 39
  7. 2022SpeechMatrix: A Large-Scale Mined Corpus of Multilingual Speech-to-Speech Translationscited by 35
  8. 2023Speech Translation with Large Language Models: An Industrial Practicecited by 28
  9. 2022IndicMT Eval: A Dataset to Meta-Evaluate Machine Translation metrics for Indian Languagescited by 27
  10. 2020Orthros: Non-autoregressive End-to-end Speech Translation with Dual-decodercited by 17
  11. 2019Robust Neural Machine Translation for Clean and Noisy Speech Transcriptscited by 13
  12. 2023End-to-End Speech-to-Text Translation: A Surveycited by 9
  13. 2019Breaking the Data Barrier: Towards Robust Speech Translation via Adversarial Stability Trainingcited by 7
  14. 2024CoSTA: Code-Switched Speech Translation using Aligned Speech-Text Interleavingcited by 4
  15. 2022indic-punct: An automatic punctuation restoration and inverse text normalization framework for Indic languagescited by 3
  16. 2019Coursera Corpus Mining and Multistage Fine-Tuning for Improving Lectures Translationcited by 2
  17. 2023VAKTA-SETU: A Speech-to-Speech Machine Translation Service in Select Indic Languagescited by 2

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.