# Dealing with training and test segmentation mismatch: FBK@IWSLT2021 arXiv: https://arxiv.org/abs/2106.12607 ยท HTML: https://mlsanity.com/paper/2106.12607/ ## Key facts - Cited by 6 papers in the map (data through 14 Oct 2026). - Placed in Speech and audio distillation, one of 12 directions in Knowledge distillation. - Builds on 27 of its 51 references that are in the map. - First posted on arXiv 23 Jun 2021. ## Placed in - Knowledge distillation / [Speech and audio distillation](https://mlsanity.com/distill/speech-and-audio-distillation/) ## Builds on (most cited first, 27 of 27) - [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017) - [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014) - [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016) - [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015) - [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018) - [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019) - [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019) - [Sequence-Level Knowledge Distillation](https://mlsanity.com/paper/1606.07947/) (2016) - [TED-LIUM 3: twice as much data and corpus repartition for experiments on speaker adaptation](https://mlsanity.com/paper/1805.04699/) (2018) - [fairseq S2T: Fast Speech-to-Text Modeling with fairseq](https://mlsanity.com/paper/2010.05171/) (2020) - [Sequence-to-Sequence Models Can Directly Translate Foreign Speech](https://mlsanity.com/paper/1703.08581/) (2017) - [Europarl-ST: A Multilingual Corpus For Speech Translation Of Parliamentary Debates](https://mlsanity.com/paper/1911.03167/) (2019) - [Listen and Translate: A Proof of Concept for End-to-End Speech-to-Text Translation](https://mlsanity.com/paper/1612.01744/) (2016) - [Multilingual Neural Machine Translation with Knowledge Distillation](https://mlsanity.com/paper/1902.10461/) (2019) - [Pre-training on high-resource speech recognition improves low-resource speech-to-text translation](https://mlsanity.com/paper/1809.01431/) (2018) - [End-to-End Automatic Speech Translation of Audiobooks](https://mlsanity.com/paper/1802.04200/) (2018) - [End-to-End Speech Translation with Knowledge Distillation](https://mlsanity.com/paper/1904.08075/) (2019) - [Leveraging Weakly Supervised Data to Improve End-to-End Speech-to-Text Translation](https://mlsanity.com/paper/1811.02050/) (2018) - [CoVoST: A Diverse Multilingual Speech-To-Text Translation Corpus](https://mlsanity.com/paper/2002.01320/) (2020) - [Tied Multitask Learning for Neural Speech Translation](https://mlsanity.com/paper/1802.06655/) (2018) - [A Comparative Study on End-to-end Speech to Text Translation](https://mlsanity.com/paper/1911.08870/) (2019) - [End-to-End Speech-Translation with Knowledge Distillation: FBK@IWSLT2020](https://mlsanity.com/paper/2006.02965/) (2020) - [Improving sequence-to-sequence speech recognition training with on-the-fly data augmentation](https://mlsanity.com/paper/1910.13296/) (2019) - [On Using SpecAugment for End-to-End Speech Translation](https://mlsanity.com/paper/1911.08876/) (2019) - [Beyond Voice Activity Detection: Hybrid Audio Segmentation for Direct Speech Translation](https://mlsanity.com/paper/2104.11710/) (2021) - [Contextualized Translation of Automatically Segmented Speech](https://mlsanity.com/paper/2008.02270/) (2020) - [On Knowledge Distillation for Direct Speech Translation](https://mlsanity.com/paper/2012.04964/) (2020) ## Built on by (most cited first, 6 of 6) - [Efficient yet Competitive Speech Translation: FBK@IWSLT2022](https://mlsanity.com/paper/2205.02629/) (2022) - [Don't Discard Fixed-Window Audio Segmentation in Speech-to-Text Translation](https://mlsanity.com/paper/2210.13363/) (2022) - [SegAugment: Maximizing the Utility of Speech Translation Data with Segmentation-based Augmentations](https://mlsanity.com/paper/2212.09699/) (2022) - [Direct Models for Simultaneous Translation and Automatic Subtitling: FBK@IWSLT2023](https://mlsanity.com/paper/2309.15554/) (2023) - [Dodging the Data Bottleneck: Automatic Subtitling with Automatically Segmented ST Corpora](https://mlsanity.com/paper/2209.10608/) (2022) - [How to Evaluate Speech Translation with Source-Aware Neural MT Metrics](https://mlsanity.com/paper/2511.03295/) (2025) ## Abstract (arXiv) This paper describes FBK's system submission to the IWSLT 2021 Offline Speech Translation task. We participated with a direct model, which is a Transformer-based architecture trained to translate English speech audio data into German texts. The training pipeline is characterized by knowledge distillation and a two-step fine-tuning procedure. Both knowledge distillation and the first fine-tuning step are carried out on manually segmented real and synthetic data, the latter being generated with an MT system trained on the available corpora. Differently, the second fine-tuning step is carried out on a random segmentation of the MuST-C v2 En-De dataset. Its main goal is to reduce the performance drops occurring when a speech translation model trained on manually segmented data (i.e. an ideal, sentence-like segmentation) is evaluated on automatically segmented audio (i.e. actual, more realistic testing conditions). For the same purpose, a custom hybrid segmentation procedure that accounts for both audio content (pauses) and for the length of the produced segments is applied to the test data before passing them to the system. At inference time, we compared this procedure with a baseline segmentation method based on Voice Activity Detection (VAD). Our results indicate the effectiveness of the proposed hybrid approach, shown by a reduction of the gap with manual segmentation from 8.3 to 1.4 BLEU points.