# Hierarchical Policy Optimization for Simultaneous Translation of Unbounded Speech

arXiv: https://arxiv.org/abs/2604.21045 · HTML: https://mlsanity.com/paper/2604.21045/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 45 references that are in the map.
- First posted on arXiv 22 Apr 2026.

## Builds on (most cited first, 24 of 24)

- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://mlsanity.com/paper/2104.09864/) (2021)
- [Efficient Streaming Language Models with Attention Sinks](https://mlsanity.com/paper/2309.17453/) (2023)
- [Language-agnostic BERT Sentence Embedding](https://mlsanity.com/paper/2007.01852/) (2020)
- [xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detection](https://mlsanity.com/paper/2310.10482/) (2023)
- [SimAlign: High Quality Word Alignments without Parallel Training Data using Static and Contextualized Embeddings](https://mlsanity.com/paper/2004.08728/) (2020)
- [GEMBA-MQM: Detecting Translation Quality Error Spans with GPT-4](https://mlsanity.com/paper/2310.13988/) (2023)
- [Learning to Translate in Real-time with Neural Machine Translation](https://mlsanity.com/paper/1610.00388/) (2016)
- [Learning Compact Metrics for MT](https://mlsanity.com/paper/2110.06341/) (2021)
- [MetricX-24: The Google Submission to the WMT 2024 Metrics Shared Task](https://mlsanity.com/paper/2410.03983/) (2024)
- [Scaling up COMETKIWI: Unbabel-IST 2023 Submission for the Quality Estimation Shared Task](https://mlsanity.com/paper/2309.11925/) (2023)
- [SimulMT to SimulST: Adapting Simultaneous Text Translation to End-to-End Simultaneous Speech Translation](https://mlsanity.com/paper/2011.02048/) (2020)
- [YODAS: Youtube-Oriented Dataset for Audio and Speech](https://mlsanity.com/paper/2406.00899/) (2024)
- [Over-Generation Cannot Be Rewarded: Length-Adaptive Average Lagging for Simultaneous Speech Translation](https://mlsanity.com/paper/2206.05807/) (2022)
- [M-Prometheus: A Suite of Open Multilingual LLM Judges](https://mlsanity.com/paper/2504.04953/) (2025)
- [TransLLaMa: LLM-based Simultaneous Translation System](https://mlsanity.com/paper/2402.04636/) (2024)
- [StreamAtt: Direct Streaming Speech-to-Text Translation with Attention-based Audio History Selection](https://mlsanity.com/paper/2406.06097/) (2024)
- [Towards Achieving Human Parity on End-to-end Simultaneous Speech Translation via LLM Agent](https://mlsanity.com/paper/2407.21646/) (2024)
- [SeqPO-SiMT: Sequential Policy Optimization for Simultaneous Machine Translation](https://mlsanity.com/paper/2505.20622/) (2025)
- [Efficient and Adaptive Simultaneous Speech Translation with Fully Unidirectional Architecture](https://mlsanity.com/paper/2504.11809/) (2025)
- [Tuning Large language model for End-to-end Speech Translation](https://mlsanity.com/paper/2310.02050/) (2023)
- [Exploiting Multimodal Reinforcement Learning for Simultaneous Machine Translation](https://mlsanity.com/paper/2102.11387/) (2021)
- [StreamUni: Achieving Streaming Speech Translation with a Unified Large Speech-Language Model](https://mlsanity.com/paper/2507.07803/) (2025)
- [Extending Automatic Machine Translation Evaluation to Book-Length Documents](https://mlsanity.com/paper/2509.17249/) (2025)

## Built on by (most cited first, 1 of 1)

- [Look Less, Hear Better: Jointly Rewarded GRPO for Streaming ASR](https://mlsanity.com/paper/2609.18333/) (2026)

## Abstract (arXiv)

Simultaneous speech translation (SST) generates translations while receiving partial speech input. Recent advances show that large language models (LLMs) can substantially improve SST quality, but at the cost of high computational overhead. To reduce this cost, prior work reformulates SST as a multi-turn dialogue task, enabling full reuse of the LLM's key-value (KV) cache and eliminating redundant feature recomputation. However, this approach relies on supervised fine-tuning (SFT) data in dialogue form, for which few human annotations exist, and existing synthesis methods cannot guarantee data quality. In this work, we propose a Hierarchical Policy Optimization (HPO) approach that post-train models trained on imperfect SFT data. We introduce a hierarchical reward that balances translation quality and latency objectives. Experiments on English to Chinese/German/Japanese demonstrate improvements of over +7 COMET score and +1.25 MetricX score at a latency of 1.5 seconds. Comprehensive ablation studies further validate the effectiveness of different quality rewards, hierarchical reward formulations, and segmentation strategies. Code can be found here https://github.com/owaski/HPO