# End-to-End Speech Translation with Knowledge Distillation

arXiv: https://arxiv.org/abs/1904.08075 · HTML: https://mlsanity.com/paper/1904.08075/

## Key facts

- Cited by 58 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Apr 2019.

## Built on by (most cited first, 30 of 58)

- [UnitY: Two-pass Direct Speech-to-speech Translation with Discrete Units](https://mlsanity.com/paper/2212.08055/) (2022)
- [STEMM: Self-learning with Speech-text Manifold Mixup for Speech Translation](https://mlsanity.com/paper/2203.10426/) (2022)
- [Cascade versus Direct Speech Translation: Do the Differences Still Make a Difference?](https://mlsanity.com/paper/2106.01045/) (2021)
- [Improving Speech Translation by Understanding and Learning from the Auxiliary Text Translation Task](https://mlsanity.com/paper/2107.05782/) (2021)
- [Learning Shared Semantic Space for Speech-to-Text Translation](https://mlsanity.com/paper/2105.03095/) (2021)
- [Stacked Acoustic-and-Textual Encoding: Integrating the Pre-trained Models into Speech Translation Encoders](https://mlsanity.com/paper/2105.05752/) (2021)
- [Fused Acoustic and Text Encoding for Multimodal Bilingual Pretraining and Speech Translation](https://mlsanity.com/paper/2102.05766/) (2021)
- [RealTranS: End-to-End Simultaneous Speech Translation with Convolutional Weighted-Shrinking Transformer](https://mlsanity.com/paper/2106.04833/) (2021)
- [Large-Scale Streaming End-to-End Speech Translation with Neural Transducers](https://mlsanity.com/paper/2204.05352/) (2022)
- [Learning When to Translate for Streaming Speech](https://mlsanity.com/paper/2109.07368/) (2021)
- [Source and Target Bidirectional Knowledge Distillation for End-to-end Speech Translation](https://mlsanity.com/paper/2104.06457/) (2021)
- [Revisiting End-to-End Speech-to-Text Translation From Scratch](https://mlsanity.com/paper/2206.04571/) (2022)
- [Under the Morphosyntactic Lens: A Multifaceted Evaluation of Gender Bias in Speech Translation](https://mlsanity.com/paper/2203.09866/) (2022)
- [CMOT: Cross-modal Mixup via Optimal Transport for Speech Translation](https://mlsanity.com/paper/2305.14635/) (2023)
- [Leveraging unsupervised and weakly-supervised data to improve direct speech-to-speech translation](https://mlsanity.com/paper/2203.13339/) (2022)
- [GigaST: A 10,000-hour Pseudo Speech Translation Corpus](https://mlsanity.com/paper/2204.03939/) (2022)
- [DUB: Discrete Unit Back-translation for Speech Translation](https://mlsanity.com/paper/2305.11411/) (2023)
- [Beyond Voice Activity Detection: Hybrid Audio Segmentation for Direct Speech Translation](https://mlsanity.com/paper/2104.11710/) (2021)
- [Regularizing End-to-End Speech Translation with Triangular Decomposition Agreement](https://mlsanity.com/paper/2112.10991/) (2021)
- [Understanding and Bridging the Modality Gap for Speech Translation](https://mlsanity.com/paper/2305.08706/) (2023)
- [Recent Advances in Direct Speech-to-text Translation](https://mlsanity.com/paper/2306.11646/) (2023)
- [MixSpeech: Cross-Modality Self-Learning with Audio-Visual Stream Mixup for Visual Speech Translation and Recognition](https://mlsanity.com/paper/2303.05309/) (2023)
- [The USTC-NELSLIP Systems for Simultaneous Speech Translation Task at IWSLT 2021](https://mlsanity.com/paper/2107.00279/) (2021)
- [Leveraging Pseudo-labeled Data to Improve Direct Speech-to-Speech Translation](https://mlsanity.com/paper/2205.08993/) (2022)
- [Non-Parametric Domain Adaptation for End-to-End Speech Translation](https://mlsanity.com/paper/2205.11211/) (2022)
- [Is "moby dick" a Whale or a Bird? Named Entities and Terminology in Speech Translation](https://mlsanity.com/paper/2109.07439/) (2021)
- [Non-autoregressive End-to-end Speech Translation with Parallel Autoregressive Rescoring](https://mlsanity.com/paper/2109.04411/) (2021)
- [Fast-MD: Fast Multi-Decoder End-to-End Speech Translation with Non-Autoregressive Hidden Intermediates](https://mlsanity.com/paper/2109.12804/) (2021)
- [Improving End-to-end Speech Translation by Leveraging Auxiliary Speech and Text Data](https://mlsanity.com/paper/2212.01778/) (2022)
- [ComSL: A Composite Speech-Language Model for End-to-End Speech-to-Text Translation](https://mlsanity.com/paper/2305.14838/) (2023)

## Abstract (arXiv)

End-to-end speech translation (ST), which directly translates from source language speech into target language text, has attracted intensive attentions in recent years. Compared to conventional pipeline systems, end-to-end ST models have advantages of lower latency, smaller model size and less error propagation. However, the combination of speech recognition and text translation in one model is more difficult than each of these two tasks. In this paper, we propose a knowledge distillation approach to improve ST model by transferring the knowledge from text translation model. Specifically, we first train a text translation model, regarded as a teacher model, and then ST model is trained to learn output probabilities from teacher model through knowledge distillation. Experiments on English- French Augmented LibriSpeech and English-Chinese TED corpus show that end-to-end ST is possible to implement on both similar and dissimilar language pairs. In addition, with the instruction of teacher model, end-to-end ST model can gain significant improvements by over 3.5 BLEU points.