mlsanityData to 7 Oct 2026

SimulSeamless: FBK at IWSLT 2024 Simultaneous Speech Translation

arXiv 2406.14177 ↗First posted 20 Jun 2024cs.CL cs.AI cs.SDcited by 1

Not on the map. Why?

Abstract (arXiv)

This paper describes the FBK's participation in the Simultaneous Translation Evaluation Campaign at IWSLT 2024. For this year's submission in the speech-to-text translation (ST) sub-track, we propose SimulSeamless, which is realized by combining AlignAtt and SeamlessM4T in its medium configuration. The SeamlessM4T model is used "off-the-shelf" and its simultaneous inference is enabled through the adoption of AlignAtt, a SimulST policy based on cross-attention that can be applied without any retraining or adaptation of the underlying model for the simultaneous task. We participated in all the Shared Task languages (English->{German, Japanese, Chinese}, and Czech->English), achieving acceptable or even better results compared to last year's submissions. SimulSeamless, covering more than 143 source languages and 200 target languages, is released at: https://github.com/hlt-mt/FBK-fairseq/.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 22 of its 34 references that are in the map. First posted on arXiv 20 Jun 2024.

Builds on

22 of 34 references.

References · 22

  1. 2017Attention Is All You Needcited by 59,932
  2. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  3. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  4. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  5. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,290
  6. 2023Scaling Speech Technology to 1,000+ Languagescited by 401
  7. 2023Google USM: Scaling Automatic Speech Recognition Beyond 100 Languagescited by 228
  8. 2020SimulEval: An Evaluation Toolkit for Simultaneous Translationcited by 94
  9. 2022Over-Generation Cannot Be Rewarded: Length-Adaptive Average Lagging for Simultaneous Speech Translationcited by 56
  10. 2020Low-Latency Sequence-to-Sequence Speech Recognition and Translation by Partial Hypothesis Selectioncited by 47
  11. 2023Sparks of Large Audio Models: A Survey and Outlookcited by 41
  12. 2023AlignAtt: Using Attention-based Audio-Translation Alignments as a Guide for Simultaneous Speech Translationcited by 35
10 more
  1. 2022Attention as a Guide for Simultaneous Speech Translationcited by 32
  2. 2022CUNI-KIT System for Simultaneous Speech Translation Task at IWSLT 2022cited by 30
  3. 2024Speech Translation with Speech Foundation Models and Large Language Models: What is There and What is Missing?cited by 23
  4. 2022Does Simultaneous Speech Translation need Simultaneous Models?cited by 22
  5. 2022M-Adapter: Modality Adaptation for End-to-End Speech-to-Text Translationcited by 15
  6. 2022Average Token Delay: A Latency Metric for Simultaneous Translationcited by 15
  7. 2022Efficient yet Competitive Speech Translation: FBK@IWSLT2022cited by 14
  8. 2021Towards the evaluation of automatic simultaneous speech translation from a communicative perspectivecited by 8
  9. 2023Direct Models for Simultaneous Translation and Automatic Subtitling: FBK@IWSLT2023cited by 6
  10. 2022The YiTrans End-to-End Speech Translation System for IWSLT 2022 Offline Shared Taskcited by 3

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024How "Real" is Your Real-Time Simultaneous Speech-to-Text Translation System?cited by 17