mlsanityData to 7 Oct 2026

On Knowledge Distillation for Direct Speech Translation

arXiv 2012.04964 ↗First posted 9 Dec 2020cs.CLcited by 11

Not on the map. Why?

Abstract (arXiv)

Direct speech translation (ST) has shown to be a complex task requiring knowledge transfer from its sub-tasks: automatic speech recognition (ASR) and machine translation (MT). For MT, one of the most promising techniques to transfer knowledge is knowledge distillation. In this paper, we compare the different solutions to distill knowledge in a sequence-to-sequence task like ST. Moreover, we analyze eventual drawbacks of this approach and how to alleviate them maintaining the benefits in terms of translation quality.

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). First posted on arXiv 9 Dec 2020.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 3; 2022: 3; 2023: 4; 2024: 1.

11 papers.

  1. 2022Attention as a Guide for Simultaneous Speech Translationcited by 32
  2. 2024StreamAtt: Direct Streaming Speech-to-Text Translation with Attention-based Audio History Selectioncited by 19
  3. 2021Beyond Voice Activity Detection: Hybrid Audio Segmentation for Direct Speech Translationcited by 17
  4. 2022Efficient yet Competitive Speech Translation: FBK@IWSLT2022cited by 14
  5. 2021Is "moby dick" a Whale or a Bird? Named Entities and Terminology in Speech Translationcited by 11
  6. 2023Tuning Large language model for End-to-end Speech Translationcited by 9
  7. 2023End-to-End Speech-to-Text Translation: A Surveycited by 9
  8. 2021Dealing with training and test segmentation mismatch: FBK@IWSLT2021cited by 6
  9. 2022Discrete Cross-Modal Alignment Enables Zero-Shot Speech Translationcited by 6
  10. 2023Direct Models for Simultaneous Translation and Automatic Subtitling: FBK@IWSLT2023cited by 6
  11. 2023How To Build Competitive Multi-gender Speech Translation Models For Controlling Speaker Gender Translationcited by 2