# End-to-End Speech-Translation with Knowledge Distillation: FBK@IWSLT2020

arXiv: https://arxiv.org/abs/2006.02965 · HTML: https://mlsanity.com/paper/2006.02965/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Jun 2020.

## Built on by (most cited first, 30 of 31)

- [STEMM: Self-learning with Speech-text Manifold Mixup for Speech Translation](https://mlsanity.com/paper/2203.10426/) (2022)
- [Cascade versus Direct Speech Translation: Do the Differences Still Make a Difference?](https://mlsanity.com/paper/2106.01045/) (2021)
- [Improving Speech Translation by Understanding and Learning from the Auxiliary Text Translation Task](https://mlsanity.com/paper/2107.05782/) (2021)
- [CTC-based Compression for Direct Speech Translation](https://mlsanity.com/paper/2102.01578/) (2021)
- [Universal Simultaneous Machine Translation with Mixture-of-Experts Wait-k Policy](https://mlsanity.com/paper/2109.05238/) (2021)
- [CMOT: Cross-modal Mixup via Optimal Transport for Speech Translation](https://mlsanity.com/paper/2305.14635/) (2023)
- [WACO: Word-Aligned Contrastive Learning for Speech Translation](https://mlsanity.com/paper/2212.09359/) (2022)
- [End-to-End Speech Translation with Pre-trained Models and Adapters: UPC at IWSLT 2021](https://mlsanity.com/paper/2105.04512/) (2021)
- [Recent Advances in Direct Speech-to-text Translation](https://mlsanity.com/paper/2306.11646/) (2023)
- [Efficient yet Competitive Speech Translation: FBK@IWSLT2022](https://mlsanity.com/paper/2205.02629/) (2022)
- [Pushing the Limits of Zero-shot End-to-End Speech Translation](https://mlsanity.com/paper/2402.10422/) (2024)
- [Is "moby dick" a Whale or a Bird? Named Entities and Terminology in Speech Translation](https://mlsanity.com/paper/2109.07439/) (2021)
- [Tuning Large language model for End-to-end Speech Translation](https://mlsanity.com/paper/2310.02050/) (2023)
- [Between Flexibility and Consistency: Joint Generation of Captions and Subtitles](https://mlsanity.com/paper/2107.06246/) (2021)
- [Dealing with training and test segmentation mismatch: FBK@IWSLT2021](https://mlsanity.com/paper/2106.12607/) (2021)
- [SegAugment: Maximizing the Utility of Speech Translation Data with Segmentation-based Augmentations](https://mlsanity.com/paper/2212.09699/) (2022)
- [An Empirical Study of Consistency Regularization for End-to-End Speech-to-Text Translation](https://mlsanity.com/paper/2308.14482/) (2023)
- [Direct Models for Simultaneous Translation and Automatic Subtitling: FBK@IWSLT2023](https://mlsanity.com/paper/2309.15554/) (2023)
- [Logit-Based Ensemble Distribution Distillation for Robust Autoregressive Sequence Uncertainties](https://mlsanity.com/paper/2305.10384/) (2023)
- [Segmenting Subtitles for Correcting ASR Segmentation Errors](https://mlsanity.com/paper/2104.07868/) (2021)
- [Improving speech translation by fusing speech and text](https://mlsanity.com/paper/2305.14042/) (2023)
- [Enhancing End-to-End Conversational Speech Translation Through Target Language Context Utilization](https://mlsanity.com/paper/2309.15686/) (2023)
- [Sentence-wise Speech Summarization: Task, Datasets, and End-to-End Modeling with LM Knowledge Distillation](https://mlsanity.com/paper/2408.00205/) (2024)
- [End-to-end Automatic Speech Recognition and Speech Translation: Integration of Speech Foundational Models and LLMs](https://mlsanity.com/paper/2510.10329/) (2025)
- [The NiuTrans End-to-End Speech Translation System for IWSLT 2021 Offline Task](https://mlsanity.com/paper/2107.02444/) (2021)
- [Efficient Speech Translation with Pre-trained Models](https://mlsanity.com/paper/2211.04939/) (2022)
- [Efficient CTC Regularization via Coarse Labels for End-to-End Speech Translation](https://mlsanity.com/paper/2302.10871/) (2023)
- [Speech Translation with Foundation Models and Optimal Transport: UPC at IWSLT23](https://mlsanity.com/paper/2306.01327/) (2023)
- [Improving End-to-End Speech Translation by Imitation-Based Knowledge Distillation with Synthetic Transcripts](https://mlsanity.com/paper/2307.08426/) (2023)
- [CTC-GMM: CTC guided modality matching for fast and accurate streaming speech translation](https://mlsanity.com/paper/2410.05146/) (2024)

## Abstract (arXiv)

This paper describes FBK's participation in the IWSLT 2020 offline speech translation (ST) task. The task evaluates systems' ability to translate English TED talks audio into German texts. The test talks are provided in two versions: one contains the data already segmented with automatic tools and the other is the raw data without any segmentation. Participants can decide whether to work on custom segmentation or not. We used the provided segmentation. Our system is an end-to-end model based on an adaptation of the Transformer for speech data. Its training process is the main focus of this paper and it is based on: i) transfer learning (ASR pretraining and knowledge distillation), ii) data augmentation (SpecAugment, time stretch and synthetic data), iii) combining synthetic and real data marked as different domains, and iv) multi-task learning using the CTC loss. Finally, after the training with word-level knowledge distillation is complete, our ST models are fine-tuned using label smoothed cross entropy. Our best model scored 29 BLEU on the MuST-C En-De test set, which is an excellent result compared to recent papers, and 23.7 BLEU on the same data segmented with VAD, showing the need for researching solutions addressing this specific data condition.