mlsanityData to 14 Oct 2026

MT-R1-Zero: Advancing LLM-based Machine Translation via R1-Zero-like Reinforcement Learning

arXiv 2504.10160 ↗First posted 14 Apr 2025cs.CL cs.AI cs.LGcited by 48

On the map: LLM reasoning → Reasoning benchmarks and evaluation

Abstract (arXiv)

Large-scale reinforcement learning (RL) methods have proven highly effective in enhancing the reasoning abilities of large language models (LLMs), particularly for tasks with verifiable solutions such as mathematics and coding. However, applying this idea to machine translation (MT), where outputs are flexibly formatted and difficult to automatically evaluate with explicit rules, remains underexplored. In this work, we introduce MT-R1-Zero, the first open-source adaptation of the R1-Zero RL framework for MT without supervised fine-tuning or cold-start. We propose a rule-metric mixed reward mechanism to guide LLMs towards improved translation quality via emergent reasoning. On the WMT 24 English-Chinese benchmark, our MT-R1-Zero-3B-Mix achieves competitive performance, surpassing TowerInstruct-7B-v0.2 by an average of 1.26 points. Meanwhile, our MT-R1-Zero-7B-Mix attains a high average score of 62.25 across all metrics, placing it on par with advanced proprietary models such as GPT-4o and Claude-3.5-Sonnet, while the MT-R1-Zero-7B-Sem variant achieves state-of-the-art scores on semantic metrics. Moreover, our work exhibits strong generalization capabilities on out-of-distribution MT tasks, robustly supporting multilingual and low-resource settings. Extensive analysis of model behavior across different initializations and reward metrics offers pioneering insight into the critical role of reward design, LLM adaptability, training dynamics, and emergent reasoning patterns within the R1-Zero paradigm for MT. Our code is available at https://github.com/fzp0424/MT-R1-Zero.

Key facts. Cited by 48 papers in the map (data through 14 Oct 2026). Placed in Reasoning benchmarks and evaluation, one of 17 directions in LLM reasoning. Builds on 38 of its 64 references that are in the map. First posted on arXiv 14 Apr 2025.

Builds on

38 of 64 references.

Closest ancestors · 4

Same direction, newest first

  1. 2025New Trends for Modern Machine Translation with Large Reasoning Models13 Mar
  2. 2025R1-T1: Fully Incentivizing Translation Capability in LLMs via Reasoning Learning27 Feb
  3. 2025Evaluating o1-Like LLMs: Unlocking Reasoning for Translation through Comprehensive Analysis17 Feb
  4. 2024Mutual Reasoning Makes Smaller LLMs Stronger Problem-Solvers12 Aug

Other references · 34

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  4. 2023Let's Verify Step by Stepcited by 3,789
  5. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,224
  6. 2025DAPO: An Open-Source LLM Reinforcement Learning System at Scalecited by 2,810
  7. 2024Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameterscited by 1,986
  8. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  9. 2024LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Modelscited by 1,464
  10. 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,312
  11. 2020COMET: A Neural Framework for MT Evaluationcited by 1,033
  12. 2025Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Modelscited by 665
22 more
  1. 2025Process Reinforcement through Implicit Rewardscited by 504
  2. 2024Improve Mathematical Reasoning in Language Models by Automated Process Supervisioncited by 441
  3. 2025Cognitive Behaviors that Enable Self-Improving Reasoners, or, Four Habits of Highly Effective STaRscited by 377
  4. 2021Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translationcited by 338
  5. 2025Demystifying Long Chain-of-Thought Reasoning in LLMscited by 326
  6. 2025rStar-Math: Small LLMs Can Master Math Reasoning with Self-Evolved Deep Thinkingcited by 313
  7. 2023Alphazero-like Tree-Search can Guide Large Language Model Decoding and Trainingcited by 295
  8. 2023xCOMET: Transparent Machine Translation Evaluation through Fine-grained Error Detectioncited by 265
  9. 2022CometKiwi: IST-Unbabel 2022 Submission for the Quality Estimation Shared Taskcited by 207
  10. 2024Tower: An Open Multilingual Large Language Model for Translation-Related Taskscited by 206
  11. 2024Free Process Rewards without Process Labelscited by 165
  12. 2024Marco-o1: Towards Open Reasoning Models for Open-Ended Solutionscited by 144
  13. 2024Aya 23: Open Weight Releases to Further Multilingual Progresscited by 123
  14. 2019On the Weaknesses of Reinforcement Learning for Neural Machine Translationcited by 82
  15. 2018A Study of Reinforcement Learning for Neural Machine Translationcited by 74
  16. 2024Scaling of Search and Learning: A Roadmap to Reproduce o1 from Reinforcement Learning Perspectivecited by 50
  17. 2024X-ALMA: Plug & Play Modules and Adaptive Rejection for Quality Translation at Scalecited by 36
  18. 2021Revisiting the Weaknesses of Reinforcement Learning for Neural Machine Translationcited by 33
  19. 2024TasTe: Teaching Large Language Models to Translate through Self-Reflectioncited by 27
  20. 2024Preliminary WMT24 Ranking of General MT Systems and LLMscited by 20
  21. 2024Ladder: A Model-Agnostic Framework Boosting LLM-based Machine Translation to the Next Levelcited by 12
  22. 2025MT-RewardTree: A Comprehensive Framework for Advancing LLM-Based Machine Translation via Reward Modelingcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 27; 2026: 21.

48 papers.

Sort:

By area:

  1. 2025Every Step Evolves: Scaling Reinforcement Learning for Trillion-Scale Thinking Modelcited by 55
  2. 2025AutoTIR: Autonomous Tools Integrated Reasoning via Reinforcement Learningcited by 33
  3. 2025100 Days After DeepSeek-R1: A Survey on Replication Studies and More Directions for Reasoning Language Modelscited by 30
  4. 2025TimeOmni-1: Incentivizing Complex Reasoning with Time Series in Large Language Modelscited by 27
  5. 2025Datasets and Recipes for Video Temporal Grounding via Reinforcement Learningcited by 19
  6. 2025SSR-Zero: Simple Self-Rewarding Reinforcement Learning for Machine Translationcited by 13
  7. 2025ReCode: Updating Code API Knowledge with Reinforcement Learningcited by 12
  8. 2025ExTrans: Multilingual Deep Reasoning Translation via Exemplar-Enhanced Reinforcement Learningcited by 11
  9. 2025R1-RE: Cross-Domain Relation Extraction with RLVRcited by 11
  10. 2025Seed LiveInterpret 2.0: End-to-end Simultaneous Speech-to-speech Translation with Your Voicecited by 9
  11. 2026Beyond Literal Mapping: Benchmarking and Improving Non-Literal Translation Evaluationcited by 6
  12. 2026TAPO: Translation Augmented Policy Optimization for Multilingual Mathematical Reasoningcited by 6
36 more
  1. 2025MT3: Scaling MLLM-based Text Image Machine Translation via Multi-Task Reinforcement Learningcited by 5
  2. 2025How Well Do Large Reasoning Models Translate? A Comprehensive Evaluation for Multi-Domain Machine Translationcited by 5
  3. 2025TACTIC: Translation Agents with Cognitive-Theoretic Interactive Collaborationcited by 5
  4. 2025RIVAL: Reinforcement Learning with Iterative and Adversarial Optimization for Machine Translationcited by 4
  5. 2025LLM Reasoning for Machine Translation: Synthetic Data Generation over Thinking Tokenscited by 4
  6. 2025M2PO: Multi-Perspective Multi-Pair Preference Optimization for Machine Translationcited by 4
  7. 2026Mending the Holes: Mitigating Reward Hacking in Reinforcement Learning for Multilingual Translationcited by 4
  8. 2025DMDTEval: An Evaluation and Analysis of LLMs on Disambiguation in Multi-domain Translationcited by 3
  9. 2025TAT-R1: Terminology-Aware Translation with Reinforcement Learning and Word Alignmentcited by 3
  10. 2025Investigating Test-Time Scaling with Reranking for Machine Translationcited by 3
  11. 2026NeoAMT: Neologism-Aware Agentic Machine Translation with Reinforcement Learningcited by 3
  12. 2026Improving Low-Resource Machine Translation via Round-Trip Reinforcement Learningcited by 3
  13. 2026LQM: Linguistically Motivated Multidimensional Quality Metrics for Machine Translationcited by 3
  14. 2025Universal Reasoner: A Single, Composable Plug-and-Play Reasoner for Frozen LLMscited by 2
  15. 2025AttriLens-Mol: Attribute Guided Reinforcement Learning for Molecular Property Prediction with Large Language Modelscited by 2
  16. 2025SUMMA: A Multimodal Large Language Model for Advertisement Summarizationcited by 2
  17. 2026GRRM: Group Relative Reward Modeling for Machine Translationcited by 2
  18. 2026Beyond Literal Translation: Evaluating Cultural Effectiveness in Social Media UGCcited by 2
  19. 2025LLaMAX2: Your Translation-Enhanced Model also Performs Well in Reasoningcited by 1
  20. 2025Remedy-R: Generative Reasoning for Machine Translation Evaluation without Error Annotationscited by 1
  21. 2026Consensus Group Relative Policy Optimization for Text Generationcited by 1
  22. 2026PEGRL: Improving Machine Translation by Post-Editing Guided Reinforcement Learningcited by 1
  23. 2026Incentivizing Parametric Knowledge via Reinforcement Learning with Verifiable Rewards for Cross-Cultural Entity Translationcited by 1
  24. 2026Retrieval, Reward, and Training Protocols: What Matters in Training Search Agents?cited by 1
  25. 2026STAR : Sentence Translation Alignment Rate for Document-to-Document Machine Translationcited by 1
  26. 2025TimeHC-RL: Temporal-aware Hierarchical Cognitive Reinforcement Learning for Enhancing LLMs' Social Intelligencecited by 0
  27. 2025A Fast Anti-Jamming Cognitive Radar Deployment Algorithm Based on Reinforcement Learningcited by 0
  28. 2025A First-Order Logic-Based Alternative to Reward Models in RLHFcited by 0
  29. 2026Beyond Scalar Scores: Reinforcement Learning for Error-Aware Quality Estimation of Machine Translationcited by 0
  30. 2026Reinforcement Learning Elicits Contextual Learning of Unseen Language Translationcited by 0
  31. 2026Reasoning Before Translation: Enhancing Legal Machine Translation with Structured Reasoningcited by 0
  32. 2026Translation with Thought: Difficulty-Adaptive Reasoning via Reinforcement Learning for Multi-Domain Machine Translationcited by 0
  33. 2026PAMT: Process-Aligned Reinforcement Learning for Multi-Domain Machine Translationcited by 0
  34. 2026Reasoning about In-Context Samples for Machine-Translationcited by 0
  35. 2026NE-R1: Enhancing Named Entity Recognition Model via Reinforcement Learningcited by 0
  36. 2026When Metrics Reward the Worst Translations: Internalizing Cultural Reasoning for Social Media Translation Evaluationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Translate-R1: Cost-Aware Translation Tool Use via Reinforcement Learning
  2. 2025Tool Zero: Training Tool-Augmented LLMs via Pure RL from Scratch
  3. 2024Advancing Translation Preference Modeling with RLHF: A Step Towards Cost-Effective Solution
  4. 2026Rewrite to Translate, Translate to Reward: Reinforcement Learning for Source Rewriting in Machine Translation
  5. 2025Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecycle
  6. 2026Reference-Free Post-Training of Open Large Language Models for Multilingual Machine Translation
  7. 2025DeepTrans: Deep Reasoning Translation via Reinforcement Learning
  8. 2026Reference-Free Reinforcement Learning Fine-Tuning for MT: A Seq2Seq Perspective