mlsanityData to 7 Oct 2026

Finetuning End-to-End Models for Estonian Conversational Spoken Language Translation

arXiv 2407.03809 ↗First posted 4 Jul 2024cs.CL eess.AScited by 1

Not on the map. Why?

Abstract (arXiv)

This paper investigates the finetuning of end-to-end models for bidirectional Estonian-English and Estonian-Russian conversational speech-to-text translation. Due to the limited availability of speech translation data for Estonian, we created additional training data by web scraping and synthesizing data from speech recognition datasets using machine translation. We evaluated three publicly available end-to-end models: Whisper, OWSM 3.1, and SeamlessM4T. Our results indicate that fine-tuning with synthetic data enhances translation accuracy by a large margin, with SeamlessM4T matching or surpassing cascaded speech translation systems that use state-of-the-art speech recognition and machine translation models.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 16 of its 22 references that are in the map. First posted on arXiv 4 Jul 2024.

Builds on

16 of 22 references.

References · 16

  1. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  2. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,290
  3. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,196
  4. 2018Averaging Weights Leads to Wider Optima and Better Generalizationcited by 1,194
  5. 2018ESPnet: End-to-End Speech Processing Toolkitcited by 554
  6. 2022FLEURS: Few-shot Learning Evaluation of Universal Representations of Speechcited by 458
  7. 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 455
  8. 2021GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audiocited by 391
  9. 2023Seamless: Multilingual Expressive and Streaming Speech Translationcited by 190
  10. 2023Powerset multi-class cross entropy loss for neural speaker diarizationcited by 109
  11. 2023SONAR: Sentence-Level Multimodal and Language-Agnostic Representationscited by 105
  12. 2021Learning Compact Metrics for MTcited by 93
4 more
  1. 2022E-Branchformer: Branchformer with Enhanced merging for speech recognitioncited by 90
  2. 2020CoVoST 2 and Massively Multilingual Speech-to-Text Translationcited by 73
  3. 2023Prompting the Hidden Talent of Web-Scale Speech Models for Zero-Shot Task Generalizationcited by 43
  4. 2020The European Language Technology Landscape in 2020: Language-Centric and Human-Centric AI for Cross-Cultural Communication in Multilingual Europecited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Automated Gradient-Driven Parameter Sharing for Low-Resource Multilingual Speech-to-Text Translationcited by 0