Efficient yet Competitive Speech Translation: FBK@IWSLT2022
Not on the map. Why?
Abstract (arXiv)
The primary goal of this FBK's systems submission to the IWSLT 2022 offline and simultaneous speech translation tasks is to reduce model training costs without sacrificing translation quality. As such, we first question the need of ASR pre-training, showing that it is not essential to achieve competitive results. Second, we focus on data filtering, showing that a simple method that looks at the ratio between source and target characters yields a quality improvement of 1 BLEU. Third, we compare different methods to reduce the detrimental effect of the audio segmentation mismatch between training data manually segmented at sentence level and inference data that is automatically segmented. Towards the same goal of training cost reduction, we participate in the simultaneous task with the same model trained for offline ST. The effectiveness of our lightweight training strategy is shown by the high score obtained on the MuST-C en-de corpus (26.7 BLEU) and is confirmed in high-resource data conditions by a 1.6 BLEU improvement on the IWSLT2020 test set over last year's winning system.
Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 34 of its 50 references that are in the map. First posted on arXiv 5 May 2022.
Builds on
34 of 50 references.
References · 34
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
- 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
- 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
- 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
- 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,242
- 2016Sequence-Level Knowledge Distillationcited by 761
- 2021VoxPopuli: A Large-Scale Multilingual Speech Corpus for Representation Learning, Semi-Supervised Learning and Interpretationcited by 455
- 2018TED-LIUM 3: twice as much data and corpus repartition for experiments on speaker adaptationcited by 224
- 2020fairseq S2T: Fast Speech-to-Text Modeling with fairseqcited by 174
- 2021Primer: Searching for Efficient Transformers for Language Modelingcited by 162
- 2019Europarl-ST: A Multilingual Corpus For Speech Translation Of Parliamentary Debatescited by 114
22 more
- 2020SimulEval: An Evaluation Toolkit for Simultaneous Translationcited by 94
- 2020SimulMT to SimulST: Adapting Simultaneous Text Translation to End-to-End Simultaneous Speech Translationcited by 80
- 2018Pre-training on high-resource speech recognition improves low-resource speech-to-text translationcited by 70
- 2020CoVoST: A Diverse Multilingual Speech-To-Text Translation Corpuscited by 57
- 2021Facebook AI WMT21 News Translation Task Submissioncited by 57
- 2021Improving Speech Translation by Understanding and Learning from the Auxiliary Text Translation Taskcited by 54
- 2021Learning Shared Semantic Space for Speech-to-Text Translationcited by 49
- 2021Stacked Acoustic-and-Textual Encoding: Integrating the Pre-trained Models into Speech Translation Encoderscited by 48
- 2021CTC-based Compression for Direct Speech Translationcited by 45
- 2019A Comparative Study on End-to-end Speech to Text Translationcited by 40
- 2021RealTranS: End-to-End Simultaneous Speech Translation with Convolutional Weighted-Shrinking Transformercited by 39
- 2020End-to-End Speech-Translation with Knowledge Distillation: FBK@IWSLT2020cited by 31
- 2021Source and Target Bidirectional Knowledge Distillation for End-to-end Speech Translationcited by 27
- 2022SHAS: Approaching optimal Segmentation for End-to-End Speech Translationcited by 26
- 2022Does Simultaneous Speech Translation need Simultaneous Models?cited by 22
- 2021Beyond Voice Activity Detection: Hybrid Audio Segmentation for Direct Speech Translationcited by 17
- 2021Speechformer: Reducing Information Loss in Direct Speech Translationcited by 15
- 2020Tight Integrated End-to-End Training for Cascaded Speech Translationcited by 13
- 2020Contextualized Translation of Automatically Segmented Speechcited by 12
- 2020On Target Segmentation for Direct Speech Translationcited by 12
- 2020On Knowledge Distillation for Direct Speech Translationcited by 11
- 2021Dealing with training and test segmentation mismatch: FBK@IWSLT2021cited by 6
Built on by
Citing papers per year: 2022: 5; 2023: 4; 2024: 3; 2025: 2.
14 papers.
Sort:
- 2022Over-Generation Cannot Be Rewarded: Length-Adaptive Average Lagging for Simultaneous Speech Translationcited by 56
- 2022Attention as a Guide for Simultaneous Speech Translationcited by 32
- 2024StreamAtt: Direct Streaming Speech-to-Text Translation with Attention-based Audio History Selectioncited by 19
- 2024How "Real" is Your Real-Time Simultaneous Speech-to-Text Translation System?cited by 17
- 2023When Good and Reproducible Results are a Giant with Feet of Clay: The Importance of Software Quality in NLPcited by 14
- 2022MT Metrics Correlate with Human Ratings of Simultaneous Speech Translationcited by 9
- 2022Don't Discard Fixed-Window Audio Segmentation in Speech-to-Text Translationcited by 8
- 2022Direct Speech Translation for Automatic Subtitlingcited by 7
- 2023Direct Models for Simultaneous Translation and Automatic Subtitling: FBK@IWSLT2023cited by 6
- 2025Prepending or Cross-Attention for Speech-to-Text? An Empirical Comparisoncited by 4
- 2025FAMA: The First Large-Scale Open-Science Speech Foundation Model for English and Italiancited by 2
- 2023Efficient CTC Regularization via Coarse Labels for End-to-End Speech Translationcited by 1
- 2025FAMA: The First Large-Scale Open-Science Speech Foundation Model for English and Italian28 May 2025
- 2025Prepending or Cross-Attention for Speech-to-Text? An Empirical Comparison4 Jan 2025
- 2024How "Real" is Your Real-Time Simultaneous Speech-to-Text Translation System?24 Dec 2024
- 2024SimulSeamless: FBK at IWSLT 2024 Simultaneous Speech Translation20 Jun 2024
- 2024StreamAtt: Direct Streaming Speech-to-Text Translation with Attention-based Audio History Selection10 Jun 2024
- 2023Direct Models for Simultaneous Translation and Automatic Subtitling: FBK@IWSLT202327 Sep 2023
- 2023Speech Translation with Foundation Models and Optimal Transport: UPC at IWSLT232 Jun 2023
- 2023When Good and Reproducible Results are a Giant with Feet of Clay: The Importance of Software Quality in NLP28 Mar 2023
- 2023Efficient CTC Regularization via Coarse Labels for End-to-End Speech Translation21 Feb 2023
- 2022Attention as a Guide for Simultaneous Speech Translation15 Dec 2022
- 2022MT Metrics Correlate with Human Ratings of Simultaneous Speech Translation16 Nov 2022
- 2022Don't Discard Fixed-Window Audio Segmentation in Speech-to-Text Translation24 Oct 2022
- 2022Over-Generation Cannot Be Rewarded: Length-Adaptive Average Lagging for Simultaneous Speech Translationcited by 56
- 2022Attention as a Guide for Simultaneous Speech Translationcited by 32
- 2024StreamAtt: Direct Streaming Speech-to-Text Translation with Attention-based Audio History Selectioncited by 19
- 2024How "Real" is Your Real-Time Simultaneous Speech-to-Text Translation System?cited by 17
- 2023When Good and Reproducible Results are a Giant with Feet of Clay: The Importance of Software Quality in NLPcited by 14
- 2022MT Metrics Correlate with Human Ratings of Simultaneous Speech Translationcited by 9
- 2022Don't Discard Fixed-Window Audio Segmentation in Speech-to-Text Translationcited by 8
- 2022Direct Speech Translation for Automatic Subtitlingcited by 7
- 2023Direct Models for Simultaneous Translation and Automatic Subtitling: FBK@IWSLT2023cited by 6
- 2025Prepending or Cross-Attention for Speech-to-Text? An Empirical Comparisoncited by 4
- 2025FAMA: The First Large-Scale Open-Science Speech Foundation Model for English and Italiancited by 2
- 2023Efficient CTC Regularization via Coarse Labels for End-to-End Speech Translationcited by 1
12 most cited of 14 All 14 (JSON)
- 2025FAMA: The First Large-Scale Open-Science Speech Foundation Model for English and Italian28 May 2025
- 2025Prepending or Cross-Attention for Speech-to-Text? An Empirical Comparison4 Jan 2025
- 2024How "Real" is Your Real-Time Simultaneous Speech-to-Text Translation System?24 Dec 2024
- 2024SimulSeamless: FBK at IWSLT 2024 Simultaneous Speech Translation20 Jun 2024
- 2024StreamAtt: Direct Streaming Speech-to-Text Translation with Attention-based Audio History Selection10 Jun 2024
- 2023Direct Models for Simultaneous Translation and Automatic Subtitling: FBK@IWSLT202327 Sep 2023
- 2023Speech Translation with Foundation Models and Optimal Transport: UPC at IWSLT232 Jun 2023
- 2023When Good and Reproducible Results are a Giant with Feet of Clay: The Importance of Software Quality in NLP28 Mar 2023
- 2023Efficient CTC Regularization via Coarse Labels for End-to-End Speech Translation21 Feb 2023
- 2022Attention as a Guide for Simultaneous Speech Translation15 Dec 2022
- 2022MT Metrics Correlate with Human Ratings of Simultaneous Speech Translation16 Nov 2022
- 2022Don't Discard Fixed-Window Audio Segmentation in Speech-to-Text Translation24 Oct 2022
12 newest of 14 All 14 (JSON)