# Decision Attentive Regularization to Improve Simultaneous Speech Translation Systems

arXiv: https://arxiv.org/abs/2110.15729 · HTML: https://mlsanity.com/paper/2110.15729/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 25 references that are in the map.
- First posted on arXiv 13 Oct 2021.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [SpecAugment: A Simple Data Augmentation Method for Automatic Speech Recognition](https://mlsanity.com/paper/1904.08779/) (2019)
- [Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates](https://mlsanity.com/paper/1804.10959/) (2018)
- [Facebook FAIR's WMT19 News Translation Task Submission](https://mlsanity.com/paper/1907.06616/) (2019)
- [Monotonic Infinite Lookback Attention for Simultaneous Machine Translation](https://mlsanity.com/paper/1906.05218/) (2019)
- [Monotonic Multihead Attention](https://mlsanity.com/paper/1909.12406/) (2019)
- [Online and Linear-Time Attention by Enforcing Monotonic Alignments](https://mlsanity.com/paper/1704.00784/) (2017)
- [Monotonic Chunkwise Attention](https://mlsanity.com/paper/1712.05382/) (2017)
- [SimulMT to SimulST: Adapting Simultaneous Text Translation to End-to-End Simultaneous Speech Translation](https://mlsanity.com/paper/2011.02048/) (2020)
- [Pre-training on high-resource speech recognition improves low-resource speech-to-text translation](https://mlsanity.com/paper/1809.01431/) (2018)
- [End-to-End Automatic Speech Translation of Audiobooks](https://mlsanity.com/paper/1802.04200/) (2018)
- [End-to-End Speech Translation with Knowledge Distillation](https://mlsanity.com/paper/1904.08075/) (2019)
- [Improving Speech Translation by Understanding and Learning from the Auxiliary Text Translation Task](https://mlsanity.com/paper/2107.05782/) (2021)
- [Thinking Slow about Latency Evaluation for Simultaneous Machine Translation](https://mlsanity.com/paper/1906.00048/) (2019)
- [Pretraining by Backtranslation for End-to-end ASR in Low-Resource Settings](https://mlsanity.com/paper/1812.03919/) (2018)

## Built on by (most cited first, 6 of 6)

- [Attention as a Guide for Simultaneous Speech Translation](https://mlsanity.com/paper/2212.07850/) (2022)
- [Does Simultaneous Speech Translation need Simultaneous Models?](https://mlsanity.com/paper/2204.03783/) (2022)
- [Blockwise Streaming Transformer for Spoken Language Understanding and Simultaneous Speech Translation](https://mlsanity.com/paper/2204.08920/) (2022)
- [Adapting Offline Speech Translation Models for Streaming with Future-Aware Distillation and Inference](https://mlsanity.com/paper/2303.07914/) (2023)
- [Recent Advances in End-to-End Simultaneous Speech Translation](https://mlsanity.com/paper/2406.00497/) (2024)
- [Direct Simultaneous Translation Activation for Large Audio-Language Models](https://mlsanity.com/paper/2509.15692/) (2025)

## Abstract (arXiv)

Simultaneous translation systems start producing the output while processing the partial source sentence in the incoming input stream. These systems need to decide when to read more input and when to write the output. These decisions depend on the structure of source/target language and the information contained in the partial input sequence. Hence, read/write decision policy remains the same across different input modalities, i.e., speech and text. This motivates us to leverage the text transcripts corresponding to the speech input for improving simultaneous speech-to-text translation (SimulST). We propose Decision Attentive Regularization (DAR) to improve the decision policy of SimulST systems by using the simultaneous text-to-text translation (SimulMT) task. We also extend several techniques from the offline speech translation domain to explore the role of SimulMT task in improving SimulST performance. Overall, we achieve 34.66% / 4.5 BLEU improvement over the baseline model across different latency regimes for the MuST-C English-German (EnDe) SimulST task.