mlsanityData to 14 Oct 2026

VAKTA-SETU: A Speech-to-Speech Machine Translation Service in Select Indic Languages

arXiv 2305.12518 ↗First posted 21 May 2023cs.CLcited by 2

Not on the map. Why?

Abstract (arXiv)

In this work, we present our deployment-ready Speech-to-Speech Machine Translation (SSMT) system for English-Hindi, English-Marathi, and Hindi-Marathi language pairs. We develop the SSMT system by cascading Automatic Speech Recognition (ASR), Disfluency Correction (DC), Machine Translation (MT), and Text-to-Speech Synthesis (TTS) models. We discuss the challenges faced during the research and development stage and the scalable deployment of the SSMT system as a publicly accessible web service. On the MT part of the pipeline too, we create a Text-to-Text Machine Translation (TTMT) service in all six translation directions involving English, Hindi, and Marathi. To mitigate data scarcity, we develop a LaBSE-based corpus filtering tool to select high-quality parallel sentences from a noisy pseudo-parallel corpus for training the TTMT system. All the data used for training the SSMT and TTMT systems and the best models are being made publicly available. Users of our system are (a) Govt. of India in the context of its new education policy (NEP), (b) tourists who criss-cross the multilingual landscape of India, (c) Indian Judiciary where a leading cause of the pendency of cases (to the order of 10 million as on date) is the translation of case papers, (d) farmers who need weather and price information and so on. We also share the feedback received from various stakeholders when our SSMT and TTMT systems were demonstrated in large public events.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 29 of its 57 references that are in the map. First posted on arXiv 21 May 2023.

Builds on

29 of 57 references.

References · 29

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  3. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  4. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 4,057
  5. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  6. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  7. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  8. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  9. 2016WaveNet: A Generative Model for Raw Audiocited by 1,722
  10. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,488
  11. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,384
  12. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 804
17 more
  1. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 788
  2. 2013Speech Recognition with Deep Recurrent Neural Networkscited by 669
  3. 2020Language-agnostic BERT Sentence Embeddingcited by 627
  4. 2015Listen, Attend and Spellcited by 543
  5. 2021MuRIL: Multilingual Representations for Indian Languagescited by 179
  6. 2021Direct speech-to-speech translation with discrete unitscited by 149
  7. 2021Samanantar: The Largest Publicly Available Parallel Corpora Collection for 11 Indic Languagescited by 136
  8. 2018Exploring Architectures, Data and Units For Streaming End-to-End Speech Recognition with RNN-Transducercited by 96
  9. 2019A scalable noisy speech dataset and online subjective test frameworkcited by 48
  10. 2021Chunked Autoregressive GAN for Conditional Waveform Synthesiscited by 38
  11. 2019Pivot-based Transfer Learning for Neural Machine Translation between Non-English Languagescited by 36
  12. 2020Improving Disfluency Detection by Self-Training a Self-Attentive Modelcited by 25
  13. 2021CLSRIL-23: Cross Lingual Speech Representations for Indic Languagescited by 22
  14. 2018Disfluency Detection using a Noisy Channel Model and a Deep Neural Language Modelcited by 11
  15. 2022LARD: Large-scale Artificial Disfluency Generationcited by 7
  16. 2023Improving Machine Translation with Phrase Pair Injection and Corpus Filteringcited by 7
  17. 2022Low-Resource End-to-end Sanskrit TTS using Tacotron2, WaveGlow and Transfer Learningcited by 2

Built on by

Cited by year

Citing papers per year: 2024: 2.

2 papers.

  1. 2024Towards Building Large Scale Datasets and State-of-the-Art Automatic Speech Translation Systems for 14 Indian Languagescited by 5
  2. 2024Attempt Towards Stress Transfer in Speech-to-Speech Machine Translationcited by 1