# Hidden Markov Transformer for Simultaneous Machine Translation

arXiv: https://arxiv.org/abs/2303.00257 · HTML: https://mlsanity.com/paper/2303.00257/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 35 references that are in the map.
- First posted on arXiv 1 Mar 2023.

## Builds on (most cited first, 24 of 24)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [Monotonic Infinite Lookback Attention for Simultaneous Machine Translation](https://mlsanity.com/paper/1906.05218/) (2019)
- [Learning to Translate in Real-time with Neural Machine Translation](https://mlsanity.com/paper/1610.00388/) (2016)
- [Monotonic Multihead Attention](https://mlsanity.com/paper/1909.12406/) (2019)
- [Can neural machine translation do simultaneous translation?](https://mlsanity.com/paper/1606.02012/) (2016)
- [Online and Linear-Time Attention by Enforcing Monotonic Alignments](https://mlsanity.com/paper/1704.00784/) (2017)
- [Efficient Wait-k Models for Simultaneous Machine Translation](https://mlsanity.com/paper/2005.08595/) (2020)
- [Incremental Decoding and Training Methods for Simultaneous Translation in Neural Machine Translation](https://mlsanity.com/paper/1806.03661/) (2018)
- [Simultaneous Translation Policies: From Fixed to Adaptive](https://mlsanity.com/paper/2004.13169/) (2020)
- [Information-Transport-based Policy for Simultaneous Translation](https://mlsanity.com/paper/2210.12357/) (2022)
- [Simpler and Faster Learning of Adaptive Policies for Simultaneous Translation](https://mlsanity.com/paper/1909.01559/) (2019)
- [Universal Simultaneous Machine Translation with Mixture-of-Experts Wait-k Policy](https://mlsanity.com/paper/2109.05238/) (2021)
- [Future-Guided Incremental Transformer for Simultaneous Translation](https://mlsanity.com/paper/2012.12465/) (2020)
- [Simultaneous Translation with Flexible Policy via Restricted Imitation Learning](https://mlsanity.com/paper/1906.01135/) (2019)
- [Modeling Dual Read/Write Paths for Simultaneous Machine Translation](https://mlsanity.com/paper/2203.09163/) (2022)
- [Modeling Concentrated Cross-Attention for Neural Machine Translation with Gaussian Mixture Model](https://mlsanity.com/paper/2109.05244/) (2021)
- [Gaussian Multi-head Attention for Simultaneous Machine Translation](https://mlsanity.com/paper/2203.09072/) (2022)
- [Reducing Position Bias in Simultaneous Machine Translation with Length-Aware Framework](https://mlsanity.com/paper/2203.09053/) (2022)
- [Wait-info Policy: Balancing Source and Target at Information Level for Simultaneous Machine Translation](https://mlsanity.com/paper/2210.11220/) (2022)
- [Turning Fixed to Adaptive: Integrating Post-Evaluation into Simultaneous Machine Translation](https://mlsanity.com/paper/2210.11900/) (2022)
- [Opportunistic Decoding with Timely Correction for Simultaneous Translation](https://mlsanity.com/paper/2005.00675/) (2020)
- [Temporally Correlated Task Scheduling for Sequence Learning](https://mlsanity.com/paper/2007.05290/) (2020)

## Built on by (most cited first, 25 of 25)

- [TransLLaMa: LLM-based Simultaneous Translation System](https://mlsanity.com/paper/2402.04636/) (2024)
- [End-to-End Simultaneous Speech Translation with Differentiable Segmentation](https://mlsanity.com/paper/2305.16093/) (2023)
- [StreamSpeech: Simultaneous Speech-to-Speech Translation with Multi-task Learning](https://mlsanity.com/paper/2406.03049/) (2024)
- [Learning Optimal Policy for Simultaneous Machine Translation via Binary Search](https://mlsanity.com/paper/2305.12774/) (2023)
- [SiLLM: Large Language Models for Simultaneous Machine Translation](https://mlsanity.com/paper/2402.13036/) (2024)
- [SeqPO-SiMT: Sequential Policy Optimization for Simultaneous Machine Translation](https://mlsanity.com/paper/2505.20622/) (2025)
- [Non-autoregressive Streaming Transformer for Simultaneous Translation](https://mlsanity.com/paper/2310.14883/) (2023)
- [Decoder-only Streaming Transformer for Simultaneous Translation](https://mlsanity.com/paper/2406.03878/) (2024)
- [Glancing Future for Simultaneous Machine Translation](https://mlsanity.com/paper/2309.06179/) (2023)
- [Simultaneous Machine Translation with Tailored Reference](https://mlsanity.com/paper/2310.13588/) (2023)
- [Adaptive Policy with Wait-k Model for Simultaneous Translation](https://mlsanity.com/paper/2310.14853/) (2023)
- [LLM as Effective Streaming Processor: Bridging Streaming-Batch Mismatches with Group Position Encoding](https://mlsanity.com/paper/2505.16983/) (2025)
- [Seed LiveInterpret 2.0: End-to-end Simultaneous Speech-to-speech Translation with Your Voice](https://mlsanity.com/paper/2507.17527/) (2025)
- [SimulPL: Aligning Human Preferences in Simultaneous Machine Translation](https://mlsanity.com/paper/2502.00634/) (2025)
- [Unified Segment-to-Segment Framework for Simultaneous Sequence Generation](https://mlsanity.com/paper/2310.17940/) (2023)
- [Self-Modifying State Modeling for Simultaneous Machine Translation](https://mlsanity.com/paper/2406.02237/) (2024)
- [LLMs Can Achieve High-quality Simultaneous Machine Translation as Efficiently as Offline](https://mlsanity.com/paper/2504.09570/) (2025)
- [Multi-Stream LLMs: Unblocking Language Models with Parallel Streams of Thoughts, Inputs and Outputs](https://mlsanity.com/paper/2605.12460/) (2026)
- [Enhanced Simultaneous Machine Translation with Word-level Policies](https://mlsanity.com/paper/2310.16417/) (2023)
- [Agent-SiMT: Agent-assisted Simultaneous Machine Translation with Large Language Models](https://mlsanity.com/paper/2406.06910/) (2024)
- [PsFuture: A Pseudo-Future-based Zero-Shot Adaptive Policy for Simultaneous Machine Translation](https://mlsanity.com/paper/2410.04075/) (2024)
- [SimulMEGA: MoE Routers are Advanced Policy Makers for Simultaneous Speech Translation](https://mlsanity.com/paper/2509.01200/) (2025)
- [Anticipating Future with Large Language Model for Simultaneous Machine Translation](https://mlsanity.com/paper/2410.22499/) (2024)
- [Dual Filter: A Transformer-like Inference Architecture for Hidden Markov Models](https://mlsanity.com/paper/2505.00818/) (2025)
- [Towards Efficient Simultaneous Inverse Text Normalization with Pretrained Text-to-Text Language Model and Read-Tag-Write Policy](https://mlsanity.com/paper/2610.19492/) (2026)

## Abstract (arXiv)

Simultaneous machine translation (SiMT) outputs the target sequence while receiving the source sequence, and hence learning when to start translating each target token is the core challenge for SiMT task. However, it is non-trivial to learn the optimal moment among many possible moments of starting translating, as the moments of starting translating always hide inside the model and can only be supervised with the observed target sequence. In this paper, we propose a Hidden Markov Transformer (HMT), which treats the moments of starting translating as hidden events and the target sequence as the corresponding observed events, thereby organizing them as a hidden Markov model. HMT explicitly models multiple moments of starting translating as the candidate hidden events, and then selects one to generate the target token. During training, by maximizing the marginal likelihood of the target sequence over multiple moments of starting translating, HMT learns to start translating at the moments that target tokens can be generated more accurately. Experiments on multiple SiMT benchmarks show that HMT outperforms strong baselines and achieves state-of-the-art performance.