mlsanityData to 7 Oct 2026

Decision Attentive Regularization to Improve Simultaneous Speech Translation Systems

arXiv 2110.15729 ↗First posted 13 Oct 2021cs.SD cs.CL eess.AScited by 6

Not on the map. Why?

Abstract (arXiv)

Simultaneous translation systems start producing the output while processing the partial source sentence in the incoming input stream. These systems need to decide when to read more input and when to write the output. These decisions depend on the structure of source/target language and the information contained in the partial input sequence. Hence, read/write decision policy remains the same across different input modalities, i.e., speech and text. This motivates us to leverage the text transcripts corresponding to the speech input for improving simultaneous speech-to-text translation (SimulST). We propose Decision Attentive Regularization (DAR) to improve the decision policy of SimulST systems by using the simultaneous text-to-text translation (SimulMT) task. We also extend several techniques from the offline speech translation domain to explore the role of SimulMT task in improving SimulST performance. Overall, we achieve 34.66% / 4.5 BLEU improvement over the baseline model across different latency regimes for the MuST-C English-German (EnDe) SimulST task.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 18 of its 25 references that are in the map. First posted on arXiv 13 Oct 2021.

Builds on

18 of 25 references.

References · 18

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  4. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  5. 2019SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognitioncited by 1,355
  6. 2018Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidatescited by 632
  7. 2019Facebook FAIR's WMT19 News Translation Task Submissioncited by 179
  8. 2019Monotonic Infinite Lookback Attention for Simultaneous Machine Translationcited by 103
  9. 2019Monotonic Multihead Attentioncited by 91
  10. 2017Online and Linear-Time Attention by Enforcing Monotonic Alignmentscited by 85
  11. 2017Monotonic Chunkwise Attentioncited by 82
  12. 2020SimulMT to SimulST: Adapting Simultaneous Text Translation to End-to-End Simultaneous Speech Translationcited by 80
6 more
  1. 2018Pre-training on high-resource speech recognition improves low-resource speech-to-text translationcited by 70
  2. 2018End-to-End Automatic Speech Translation of Audiobookscited by 65
  3. 2019End-to-End Speech Translation with Knowledge Distillationcited by 58
  4. 2021Improving Speech Translation by Understanding and Learning from the Auxiliary Text Translation Taskcited by 54
  5. 2019Thinking Slow about Latency Evaluation for Simultaneous Machine Translationcited by 28
  6. 2018Pretraining by Backtranslation for End-to-end ASR in Low-Resource Settingscited by 7

Built on by

Cited by year

Citing papers per year: 2022: 3; 2023: 1; 2024: 1; 2025: 1.

6 papers.

  1. 2022Attention as a Guide for Simultaneous Speech Translationcited by 32
  2. 2022Does Simultaneous Speech Translation need Simultaneous Models?cited by 22
  3. 2022Blockwise Streaming Transformer for Spoken Language Understanding and Simultaneous Speech Translationcited by 11
  4. 2023Adapting Offline Speech Translation Models for Streaming with Future-Aware Distillation and Inferencecited by 6
  5. 2024Recent Advances in End-to-End Simultaneous Speech Translationcited by 5
  6. 2025Direct Simultaneous Translation Activation for Large Audio-Language Modelscited by 0