mlsanityData to 14 Oct 2026

Tagged End-to-End Simultaneous Speech Translation Training using Simultaneous Interpretation Data

arXiv 2306.08582 ↗First posted 14 Jun 2023cs.CL cs.SD eess.AScited by 6

Not on the map. Why?

Abstract (arXiv)

Simultaneous speech translation (SimulST) translates partial speech inputs incrementally. Although the monotonic correspondence between input and output is preferable for smaller latency, it is not the case for distant language pairs such as English and Japanese. A prospective approach to this problem is to mimic simultaneous interpretation (SI) using SI data to train a SimulST model. However, the size of such SI data is limited, so the SI data should be used together with ordinary bilingual data whose translations are given in offline. In this paper, we propose an effective way to train a SimulST model using mixed data of SI and offline. The proposed method trains a single model using the mixed data with style tags that tell the model to generate SI- or offline-style outputs. Experiment results show improvements of BLEURT in different latency ranges, and our analyses revealed the proposed model generates SI-style outputs more than the baseline.

Key facts. Cited by 6 papers in the map (data through 14 Oct 2026). Builds on 24 of its 45 references that are in the map. First posted on arXiv 14 Jun 2023.

Builds on

24 of 45 references.

References · 24

  1. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,512
  2. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  3. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  4. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,488
  5. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,203
  6. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,035
  7. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 469
  8. 2019Libri-Light: A Benchmark for ASR with Limited or No Supervisioncited by 467
  9. 2018Adaptive Input Representations for Neural Language Modelingcited by 226
  10. 2019Europarl-ST: A Multilingual Corpus For Speech Translation Of Parliamentary Debatescited by 114
  11. 2016Learning to Translate in Real-time with Neural Machine Translationcited by 97
  12. 2018Exploring Architectures, Data and Units For Streaming End-to-End Speech Recognition with RNN-Transducercited by 96
12 more
  1. 2020SimulEval: An Evaluation Toolkit for Simultaneous Translationcited by 94
  2. 2020SimulMT to SimulST: Adapting Simultaneous Text Translation to End-to-End Simultaneous Speech Translationcited by 81
  3. 2019Tagged Back-Translationcited by 77
  4. 2020Data Augmenting Contrastive Learning of Speech Representations in the Time Domaincited by 63
  5. 2020CoVoST: A Diverse Multilingual Speech-To-Text Translation Corpuscited by 57
  6. 2018Incremental Decoding and Training Methods for Simultaneous Translation in Neural Machine Translationcited by 52
  7. 2020Low-Latency Sequence-to-Sequence Speech Recognition and Translation by Partial Hypothesis Selectioncited by 49
  8. 2016Domain Control for Neural Machine Translationcited by 36
  9. 2022CUNI-KIT System for Simultaneous Speech Translation Task at IWSLT 2022cited by 30
  10. 2022Average Token Delay: A Latency Metric for Simultaneous Translationcited by 15
  11. 2023Inter-connection: Effective Connection between Pre-trained Encoder and Decoder for Speech Translationcited by 3
  12. 2023NAIST-SIC-Aligned: an Aligned English-Japanese Simultaneous Interpretation Corpuscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 5; 2025: 0; 2026: 1.

6 papers.

  1. 2024TransLLaMa: LLM-based Simultaneous Translation Systemcited by 34
  2. 2024How "Real" is Your Real-Time Simultaneous Speech-to-Text Translation System?cited by 17
  3. 2024Recent Advances in End-to-End Simultaneous Speech Translationcited by 5
  4. 2024Simultaneous Interpretation Corpus Construction by Large Language Models in Distant Language Paircited by 3
  5. 2024Word Order in English-Japanese Simultaneous Interpretation: Analyses and Evaluation using Chunk-wise Monotonic Translationcited by 1
  6. 2026Streaming Translation and Transcription Through Speech-to-Text Causal Alignmentcited by 1