mlsanityData to 7 Oct 2026

Does Simultaneous Speech Translation need Simultaneous Models?

arXiv 2204.03783 ↗First posted 8 Apr 2022cs.CLcited by 22

Not on the map. Why?

Abstract (arXiv)

In simultaneous speech translation (SimulST), finding the best trade-off between high translation quality and low latency is a challenging task. To meet the latency constraints posed by the different application scenarios, multiple dedicated SimulST models are usually trained and maintained, generating high computational costs. In this paper, motivated by the increased social and environmental impact caused by these costs, we investigate whether a single model trained offline can serve not only the offline but also the simultaneous task without the need for any additional training or adaptation. Experiments on en->{de, es} indicate that, aside from facilitating the adoption of well-established offline techniques and architectures without affecting latency, the offline solution achieves similar or better translation quality compared to the same model trained in simultaneous settings, as well as being competitive with the SimulST state of the art.

Key facts. Cited by 22 papers in the map (data through 7 Oct 2026). Builds on 37 of its 66 references that are in the map. First posted on arXiv 8 Apr 2022.

Builds on

37 of 66 references.

References · 37

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  3. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  4. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  5. 2020Conformer: Convolution-augmented Transformer for Speech Recognitioncited by 1,838
  6. 2019Transformer-XL: Attentive Language Models Beyond a Fixed-Length Contextcited by 1,830
  7. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  8. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  9. 2019Energy and Policy Considerations for Deep Learning in NLPcited by 1,242
  10. 2016Sequence-Level Knowledge Distillationcited by 761
  11. 2019Green AIcited by 620
  12. 2015Sequence Level Training with Recurrent Neural Networkscited by 568
25 more
  1. 2020Towards the Systematic Reporting of the Energy and Carbon Footprints of Machine Learningcited by 246
  2. 2020fairseq S2T: Fast Speech-to-Text Modeling with fairseqcited by 174
  3. 2019Understanding and Improving Transformer From a Multi-Particle Dynamic System Point of Viewcited by 108
  4. 2019The role of artificial intelligence in achieving the Sustainable Development Goalscited by 100
  5. 2020SimulEval: An Evaluation Toolkit for Simultaneous Translationcited by 94
  6. 2019Monotonic Multihead Attentioncited by 91
  7. 2020ESPnet-ST: All-in-One Speech Translation Toolkitcited by 82
  8. 2020SimulMT to SimulST: Adapting Simultaneous Text Translation to End-to-End Simultaneous Speech Translationcited by 80
  9. 2020Efficient Wait-k Models for Simultaneous Machine Translationcited by 59
  10. 2020Bridging the Modality Gap for Speech-to-Text Translationcited by 55
  11. 2020Dual-decoder Transformer for Joint Automatic Speech Recognition and Multilingual Speech Translationcited by 54
  12. 2021CTC-based Compression for Direct Speech Translationcited by 45
  13. 2021RealTranS: End-to-End Simultaneous Speech Translation with Convolutional Weighted-Shrinking Transformercited by 39
  14. 2021Why does CTC result in peaky behavior?cited by 32
  15. 2021Direct Simultaneous Speech-to-Text Translation Assisted by Synchronized Streaming ASRcited by 31
  16. 2020Streaming Simultaneous Speech Translation with Augmented Memory Transformercited by 30
  17. 2019Thinking Slow about Latency Evaluation for Simultaneous Machine Translationcited by 28
  18. 2020Consecutive Decoding for Speech-to-text Translationcited by 28
  19. 2021Speechformer: Reducing Information Loss in Direct Speech Translationcited by 15
  20. 2021An Empirical Study of End-to-end Simultaneous Speech Translation Decoding Strategiescited by 14
  21. 2021The USTC-NELSLIP Systems for Simultaneous Speech Translation Task at IWSLT 2021cited by 13
  22. 2020On Target Segmentation for Direct Speech Translationcited by 12
  23. 2021Non-autoregressive End-to-end Speech Translation with Parallel Autoregressive Rescoringcited by 9
  24. 2021Decision Attentive Regularization to Improve Simultaneous Speech Translation Systemscited by 6
  25. 2021The NiuTrans End-to-End Speech Translation System for IWSLT 2021 Offline Taskcited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 5; 2024: 10; 2025: 1; 2026: 2.

22 papers.

Sort:

  1. 2022Over-Generation Cannot Be Rewarded: Length-Adaptive Average Lagging for Simultaneous Speech Translationcited by 56
  2. 2023AlignAtt: Using Attention-based Audio-Translation Alignments as a Guide for Simultaneous Speech Translationcited by 35
  3. 2022Attention as a Guide for Simultaneous Speech Translationcited by 32
  4. 2024Speech Translation with Speech Foundation Models and Large Language Models: What is There and What is Missing?cited by 23
  5. 2024StreamAtt: Direct Streaming Speech-to-Text Translation with Attention-based Audio History Selectioncited by 19
  6. 2024How "Real" is Your Real-Time Simultaneous Speech-to-Text Translation System?cited by 17
  7. 2022Efficient yet Competitive Speech Translation: FBK@IWSLT2022cited by 14
  8. 2024Simultaneous Masking, Not Prompting Optimization: A Paradigm Shift in Fine-tuning LLMs for Simultaneous Translationcited by 11
  9. 2023Simultaneous Machine Translation with Large Language Modelscited by 9
  10. 2024Simul-Whisper: Attention-Guided Streaming Whisper with Truncation Detectioncited by 7
  11. 2023Adapting Offline Speech Translation Models for Streaming with Future-Aware Distillation and Inferencecited by 6
  12. 2023Direct Models for Simultaneous Translation and Automatic Subtitling: FBK@IWSLT2023cited by 6
10 more
  1. 2022Joint Speech Translation and Named Entity Recognitioncited by 5
  2. 2024Recent Advances in End-to-End Simultaneous Speech Translationcited by 5
  3. 2024Label-Synchronous Neural Transducer for E2E Simultaneous Speech Translationcited by 5
  4. 2023Learning When to Speak: Latency and Quality Trade-offs for Simultaneous Speech-to-Speech Translation with Offline Modelscited by 2
  5. 2024PsFuture: A Pseudo-Future-based Zero-Shot Adaptive Policy for Simultaneous Machine Translationcited by 2
  6. 2025MLLP-VRAIN UPV system for the IWSLT 2025 Simultaneous Speech Translation Translation taskcited by 2
  7. 2024SimulSeamless: FBK at IWSLT 2024 Simultaneous Speech Translationcited by 1
  8. 2024NAIST Simultaneous Speech Translation System for IWSLT 2024cited by 1
  9. 2026A Pocket Offline Model for Simultaneous Speech Translation as CUNI Submission to IWSLT 2026cited by 0
  10. 2026Dual-Stream Simultaneous Translation via 2D Grid Attentioncited by 0