mlsanityData to 7 Oct 2026

End to end Hindi to English speech conversion using Bark, mBART and a finetuned XLSR Wav2Vec2

arXiv 2401.06183 ↗First posted 11 Jan 2024eess.AS cs.AI cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

Speech has long been a barrier to effective communication and connection, persisting as a challenge in our increasingly interconnected world. This research paper introduces a transformative solution to this persistent obstacle an end-to-end speech conversion framework tailored for Hindi-to-English translation, culminating in the synthesis of English audio. By integrating cutting-edge technologies such as XLSR Wav2Vec2 for automatic speech recognition (ASR), mBART for neural machine translation (NMT), and a Text-to-Speech (TTS) synthesis component, this framework offers a unified and seamless approach to cross-lingual communication. We delve into the intricate details of each component, elucidating their individual contributions and exploring the synergies that enable a fluid transition from spoken Hindi to synthesized English audio.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 11 of its 17 references that are in the map. First posted on arXiv 11 Jan 2024.

Builds on

11 of 17 references.

References · 11

  1. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  2. 2019Common Voice: A Massively-Multilingual Speech Corpuscited by 1,358
  3. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,028
  4. 2017Natural TTS Synthesis by Conditioning WaveNet on Mel Spectrogram Predictionscited by 800
  5. 2020FastSpeech 2: Fast and High-Quality End-to-End Text to Speechcited by 710
  6. 2021XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scalecited by 589
  7. 2017Tacotron: Towards End-to-End Speech Synthesiscited by 487
  8. 2018A disciplined approach to neural network hyper-parameters: Part 1 -- learning rate, batch size, momentum, and weight decaycited by 223
  9. 2021Exploring Wav2vec 2.0 fine-tuning for improved speech emotion recognitioncited by 47
  10. 2020Applying Wav2vec2.0 to Speech Recognition in Various Low-resource Languagescited by 30
  11. 2022Applying wav2vec2 for Speech Recognition on Bengali Common Voices Datasetcited by 4

Built on by

Cited by year

Citing papers per year: 2025: 1.

1 paper.

  1. 2025Who Gets the Mic? Investigating Gender Bias in the Speaker Assignment of a Speech-LLMcited by 2