# Worse WER, but Better BLEU? Leveraging Word Embedding as Intermediate in Multitask End-to-End Speech Translation

arXiv: https://arxiv.org/abs/2005.10678 · HTML: https://mlsanity.com/paper/2005.10678/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 May 2020.

## Built on by (most cited first, 12 of 12)

- [Multi-Task Learning in Natural Language Processing: An Overview](https://mlsanity.com/paper/2109.09138/) (2021)
- [Pre-training for Speech Translation: CTC Meets Optimal Transport](https://mlsanity.com/paper/2301.11716/) (2023)
- [Investigating the Reordering Capability in CTC-based Non-Autoregressive End-to-End Speech Translation](https://mlsanity.com/paper/2105.04840/) (2021)
- [Pushing the Limits of Zero-shot End-to-End Speech Translation](https://mlsanity.com/paper/2402.10422/) (2024)
- [Who Are We Talking About? Handling Person Names in Speech Translation](https://mlsanity.com/paper/2205.06755/) (2022)
- [Contrastive Learning for Task-Independent SpeechLLM-Pretraining](https://mlsanity.com/paper/2412.15712/) (2024)
- [Discrete Cross-Modal Alignment Enables Zero-Shot Speech Translation](https://mlsanity.com/paper/2210.09556/) (2022)
- [Weakly Supervised Multimodal Temporal Forgery Localization via Multitask Learning](https://mlsanity.com/paper/2508.02179/) (2025)
- [AlloST: Low-resource Speech Translation without Source Transcription](https://mlsanity.com/paper/2105.00171/) (2021)
- [Optimal Transport Regularization for Speech Text Alignment in Spoken Language Models](https://mlsanity.com/paper/2508.08131/) (2025)
- [Objective Soups: Multilingual Multi-Task Modeling for Speech Processing](https://mlsanity.com/paper/2508.09228/) (2025)
- [Listen, Attend, Understand: a Regularization Technique for Stable E2E Speech Translation Training on High Variance labels](https://mlsanity.com/paper/2601.01121/) (2026)

## Abstract (arXiv)

Speech translation (ST) aims to learn transformations from speech in the source language to the text in the target language. Previous works show that multitask learning improves the ST performance, in which the recognition decoder generates the text of the source language, and the translation decoder obtains the final translations based on the output of the recognition decoder. Because whether the output of the recognition decoder has the correct semantics is more critical than its accuracy, we propose to improve the multitask ST model by utilizing word embedding as the intermediate.