mlsanityData to 14 Oct 2026

SeqPO-SiMT: Sequential Policy Optimization for Simultaneous Machine Translation

arXiv 2505.20622 ↗First posted 27 May 2025cs.CL cs.AI cs.LGcited by 13

On the map: LLM alignment / RLHF / preference optimisation → LLM preference optimization methods

Abstract (arXiv)

We present Sequential Policy Optimization for Simultaneous Machine Translation (SeqPO-SiMT), a new policy optimization framework that defines the simultaneous machine translation (SiMT) task as a sequential decision making problem, incorporating a tailored reward to enhance translation quality while reducing latency. In contrast to popular Reinforcement Learning from Human Feedback (RLHF) methods, such as PPO and DPO, which are typically applied in single-step tasks, SeqPO-SiMT effectively tackles the multi-step SiMT task. This intuitive framework allows the SiMT LLMs to simulate and refine the SiMT process using a tailored reward. We conduct experiments on six datasets from diverse domains for En to Zh and Zh to En SiMT tasks, demonstrating that SeqPO-SiMT consistently achieves significantly higher translation quality with lower latency. In particular, SeqPO-SiMT outperforms the supervised fine-tuning (SFT) model by 1.13 points in COMET, while reducing the Average Lagging by 6.17 in the NEWSTEST2021 En to Zh dataset. While SiMT operates with far less context than offline translation, the SiMT results of SeqPO-SiMT on 7B LLM surprisingly rival the offline translation of high-performing LLMs, including Qwen-2.5-7B-Instruct and LLaMA-3-8B-Instruct.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Placed in LLM preference optimization methods, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 25 of its 41 references that are in the map. First posted on arXiv 27 May 2025.

Builds on

25 of 41 references.

Closest ancestors · 2

Same direction, newest first

  1. 2025SimulPL: Aligning Human Preferences in Simultaneous Machine Translation2 Feb
  2. 2022Training language models to follow instructions with human feedback4 Mar

Other references · 23

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2025DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learningcited by 6,330
  5. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,203
  6. 2020COMET: A Neural Framework for MT Evaluationcited by 1,033
  7. 2023Safe RLHF: Safe Reinforcement Learning from Human Feedbackcited by 656
  8. 2021GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audiocited by 399
  9. 2023Pretraining Language Models with Human Preferencescited by 247
  10. 2021WenetSpeech: A 10000+ Hours Multi-domain Mandarin Corpus for Speech Recognitioncited by 197
  11. 2019Monotonic Infinite Lookback Attention for Simultaneous Machine Translationcited by 104
  12. 2016Learning to Translate in Real-time with Neural Machine Translationcited by 97
11 more
  1. 2016Can neural machine translation do simultaneous translation?cited by 86
  2. 2024Error Span Annotation: A Balanced Approach for Human Evaluation of Machine Translationcited by 62
  3. 2020CoVoST: A Diverse Multilingual Speech-To-Text Translation Corpuscited by 57
  4. 2022Information-Transport-based Policy for Simultaneous Translationcited by 40
  5. 2019Simpler and Faster Learning of Adaptive Policies for Simultaneous Translationcited by 37
  6. 2024TransLLaMa: LLM-based Simultaneous Translation Systemcited by 34
  7. 2023Hidden Markov Transformer for Simultaneous Machine Translationcited by 25
  8. 2024Towards Achieving Human Parity on End-to-end Simultaneous Speech Translation via LLM Agentcited by 14
  9. 2024Decoder-only Streaming Transformer for Simultaneous Translationcited by 11
  10. 2021Exploiting Multimodal Reinforcement Learning for Simultaneous Machine Translationcited by 7
  11. 2024Self-Modifying State Modeling for Simultaneous Machine Translationcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 6; 2026: 7.

13 papers.

Sort:

By area:

  1. 2025Seed LiveInterpret 2.0: End-to-end Simultaneous Speech-to-speech Translation with Your Voicecited by 9
  2. 2025SynapseRoute: An Auto-Route Switching Framework on Dual-State Large Language Modelcited by 7
  3. 2026Simultaneous Speech-to-Speech Translation Without Aligned Datacited by 7
  4. 2025SAS: Simulated Attention Scorecited by 4
  5. 2026Unveiling the Entropy Dynamics of Chain-of-Thought Reasoningcited by 4
  6. 2026From Static Inference to Dynamic Interaction: A Survey of Streaming Large Language Modelscited by 3
  7. 2025SASST: Leveraging Syntax-Aware Chunking and LLMs for Simultaneous Speech Translationcited by 2
  8. 2025Benchmarking the Thinking Mode of Multimodal Large Language Models in Clinical Taskscited by 2
  9. 2025Diagnosing Shoulder Disorders Using Multimodal Large Language Models and Consumer-Grade Camerascited by 1
  10. 2026GeoNorm: Unify Pre-Norm and Post-Norm with Geodesic Optimizationcited by 1
  11. 2026Hierarchical Policy Optimization for Simultaneous Translation of Unbounded Speechcited by 1
  12. 2026ProactiveLLM: Learning Active Interaction for Streaming Large Language Modelscited by 0
1 more
  1. 2026NaturalFlow: Reducing Disruptive Pauses for Natural Speech Flow in Simultaneous Speech-to-Speech Translationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Reference-Free Reinforcement Learning Fine-Tuning for MT: A Seq2Seq Perspective
  2. 2021Universal Simultaneous Machine Translation with Mixture-of-Experts Wait-k Policy
  3. 2024Agent-SiMT: Agent-assisted Simultaneous Machine Translation with Large Language Models
  4. 2022Improving Simultaneous Machine Translation with Monolingual Data
  5. 2026PEGRL: Improving Machine Translation by Post-Editing Guided Reinforcement Learning
  6. 2026Reference-Free Post-Training of Open Large Language Models for Multilingual Machine Translation
  7. 2024Fine-Grained Reward Optimization for Machine Translation using Error Severity Mappings
  8. 2024QE-EBM: Using Quality Estimators as Energy Loss for Machine Translation