# Future-Guided Incremental Transformer for Simultaneous Translation

arXiv: https://arxiv.org/abs/2012.12465 · HTML: https://mlsanity.com/paper/2012.12465/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Dec 2020.

## Built on by (most cited first, 30 of 30)

- [Information-Transport-based Policy for Simultaneous Translation](https://mlsanity.com/paper/2210.12357/) (2022)
- [Universal Simultaneous Machine Translation with Mixture-of-Experts Wait-k Policy](https://mlsanity.com/paper/2109.05238/) (2021)
- [Hidden Markov Transformer for Simultaneous Machine Translation](https://mlsanity.com/paper/2303.00257/) (2023)
- [End-to-End Simultaneous Speech Translation with Differentiable Segmentation](https://mlsanity.com/paper/2305.16093/) (2023)
- [Modeling Dual Read/Write Paths for Simultaneous Machine Translation](https://mlsanity.com/paper/2203.09163/) (2022)
- [StreamSpeech: Simultaneous Speech-to-Speech Translation with Multi-task Learning](https://mlsanity.com/paper/2406.03049/) (2024)
- [Stream-Omni: Simultaneous Multimodal Interactions with Large Language-Vision-Speech Model](https://mlsanity.com/paper/2506.13642/) (2025)
- [Towards Incremental Transformers: An Empirical Analysis of Transformer Models for Incremental NLU](https://mlsanity.com/paper/2109.07364/) (2021)
- [Gaussian Multi-head Attention for Simultaneous Machine Translation](https://mlsanity.com/paper/2203.09072/) (2022)
- [Reducing Position Bias in Simultaneous Machine Translation with Length-Aware Framework](https://mlsanity.com/paper/2203.09053/) (2022)
- [Wait-info Policy: Balancing Source and Target at Information Level for Simultaneous Machine Translation](https://mlsanity.com/paper/2210.11220/) (2022)
- [Learning Optimal Policy for Simultaneous Machine Translation via Binary Search](https://mlsanity.com/paper/2305.12774/) (2023)
- [Turning Fixed to Adaptive: Integrating Post-Evaluation into Simultaneous Machine Translation](https://mlsanity.com/paper/2210.11900/) (2022)
- [Improving Simultaneous Machine Translation with Monolingual Data](https://mlsanity.com/paper/2212.01188/) (2022)
- [Glancing Future for Simultaneous Machine Translation](https://mlsanity.com/paper/2309.06179/) (2023)
- [Simultaneous Machine Translation with Tailored Reference](https://mlsanity.com/paper/2310.13588/) (2023)
- [Adaptive Policy with Wait-k Model for Simultaneous Translation](https://mlsanity.com/paper/2310.14853/) (2023)
- [Guiding Teacher Forcing with Seer Forcing for Neural Machine Translation](https://mlsanity.com/paper/2106.06751/) (2021)
- [Adapting Offline Speech Translation Models for Streaming with Future-Aware Distillation and Inference](https://mlsanity.com/paper/2303.07914/) (2023)
- [Unified Segment-to-Segment Framework for Simultaneous Sequence Generation](https://mlsanity.com/paper/2310.17940/) (2023)
- [Monotonic Simultaneous Translation with Chunk-wise Reordering and Refinement](https://mlsanity.com/paper/2110.09646/) (2021)
- [Enhanced Simultaneous Machine Translation with Word-level Policies](https://mlsanity.com/paper/2310.16417/) (2023)
- [Agent-SiMT: Agent-assisted Simultaneous Machine Translation with Large Language Models](https://mlsanity.com/paper/2406.06910/) (2024)
- [Fixed and Adaptive Simultaneous Machine Translation Strategies Using Adapters](https://mlsanity.com/paper/2407.13469/) (2024)
- [PsFuture: A Pseudo-Future-based Zero-Shot Adaptive Policy for Simultaneous Machine Translation](https://mlsanity.com/paper/2410.04075/) (2024)
- [KD4MT: A Survey of Knowledge Distillation for Machine Translation](https://mlsanity.com/paper/2602.15845/) (2026)
- [CBSiMT: Mitigating Hallucination in Simultaneous Machine Translation with Weighted Prefix-to-Prefix Training](https://mlsanity.com/paper/2311.03672/) (2023)
- [Full-Sentence Models Perform Better in Simultaneous Translation Using the Information Enhanced Decoding Strategy](https://mlsanity.com/paper/2105.01893/) (2021)
- [Context Consistency between Training and Testing in Simultaneous Machine Translation](https://mlsanity.com/paper/2311.07066/) (2023)
- [Dual-Stream Simultaneous Translation via 2D Grid Attention](https://mlsanity.com/paper/2609.34902/) (2026)

## Abstract (arXiv)

Simultaneous translation (ST) starts translations synchronously while reading source sentences, and is used in many online scenarios. The previous wait-k policy is concise and achieved good results in ST. However, wait-k policy faces two weaknesses: low training speed caused by the recalculation of hidden states and lack of future source information to guide training. For the low training speed, we propose an incremental Transformer with an average embedding layer (AEL) to accelerate the speed of calculation of the hidden states during training. For future-guided training, we propose a conventional Transformer as the teacher of the incremental Transformer, and try to invisibly embed some future information in the model through knowledge distillation. We conducted experiments on Chinese-English and German-English simultaneous translation tasks and compared with the wait-k policy to evaluate the proposed method. Our method can effectively increase the training speed by about 28 times on average at different k and implicitly embed some predictive abilities in the model, achieving better translation quality than wait-k baseline.