mlsanityData to 14 Oct 2026

Optimizing the Training Schedule of Multilingual NMT using Reinforcement Learning

arXiv 2410.06118 ↗First posted 8 Oct 2024cs.CLcited by 0

On the map: Deep reinforcement learning methods → RL training of language and agent models

Abstract (arXiv)

Multilingual NMT is a viable solution for translating low-resource languages (LRLs) when data from high-resource languages (HRLs) from the same language family is available. However, the training schedule, i.e. the order of presentation of languages, has an impact on the quality of such systems. Here, in a many-to-one translation setting, we propose to apply two algorithms that use reinforcement learning to optimize the training schedule of NMT: (1) Teacher-Student Curriculum Learning and (2) Deep Q Network. The former uses an exponentially smoothed estimate of the returns of each action based on the loss on monolingual or multilingual development subsets, while the latter estimates rewards using an additional neural network trained from the history of actions selected in different states of the system, together with the rewards received. On a 8-to-1 translation dataset with LRLs and HRLs, our second method improves BLEU and COMET scores with respect to both random selection of monolingual batches and shuffled multilingual batches, by adjusting the number of presentations of LRL vs. HRL batches.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL training of language and agent models, one of 15 directions in Deep reinforcement learning methods. Builds on 43 of its 55 references that are in the map. First posted on arXiv 8 Oct 2024.

Builds on

43 of 55 references.

References · 43

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2017Proximal Policy Optimization Algorithmscited by 15,215
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  6. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  7. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  8. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  9. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  10. 2016An overview of gradient descent optimization algorithmscited by 1,033
  11. 2015Sequence Level Training with Recurrent Neural Networkscited by 576
  12. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 469
31 more
  1. 2020Curriculum Learning for Reinforcement Learning Domains: A Framework and Surveycited by 373
  2. 2019Massively Multilingual Neural Machine Translation in the Wild: Findings and Challengescited by 257
  3. 2017OpenNMT: Open-source Toolkit for Neural Machine Translationcited by 251
  4. 2019Massively Multilingual Neural Machine Translationcited by 246
  5. 2017Teacher-Student Curriculum Learningcited by 210
  6. 2024Tower: An Open Multilingual Large Language Model for Translation-Related Taskscited by 206
  7. 2019Competence-based Curriculum Learning for Neural Machine Translationcited by 202
  8. 2020On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translationcited by 127
  9. 2018When and Why are Pre-trained Word Embeddings Useful for Neural Machine Translation?cited by 126
  10. 2018Rapid Adaptation of Neural Machine Translation to New Languagescited by 82
  11. 2018A Study of Reinforcement Learning for Neural Machine Translationcited by 74
  12. 2020Balancing Training for Multilingual Neural Machine Translationcited by 72
  13. 2018Universal Neural Machine Translation for Extremely Low Resource Languagescited by 66
  14. 2020Norm-Based Curriculum Learning for Neural Machine Translationcited by 60
  15. 2018An Empirical Exploration of Curriculum Learning for Neural Machine Translationcited by 58
  16. 2017Classical Structured Prediction Losses for Sequence to Sequence Learningcited by 55
  17. 2018Denoising Neural Machine Translation Training with Trusted Data and Online Data Selectioncited by 43
  18. 2021Revisiting the Weaknesses of Reinforcement Learning for Neural Machine Translationcited by 33
  19. 2019Reinforcement Learning based Curriculum Optimization for Neural Machine Translationcited by 29
  20. 2020Self-Paced Learning for Neural Machine Translationcited by 27
  21. 2019Adaptive Scheduling for Multi-Task Learningcited by 24
  22. 2019Multilingual Neural Machine Translation With Soft Decoupled Encodingcited by 21
  23. 2020Reinforced Curriculum Learning on Pre-trained Neural Machine Translation Modelscited by 20
  24. 2021Uncertainty-Aware Balancing for Multilingual and Multi-Domain Neural Machine Translation Trainingcited by 19
  25. 2019Learning a Multi-Domain Curriculum for Neural Machine Translationcited by 16
  26. 2021Bandits Don't Follow Rules: Balancing Multi-Facet Machine Translation with Multi-Armed Banditscited by 16
  27. 2023ESRL: Efficient Sampling-based Reinforcement Learning for Sequence Generationcited by 15
  28. 2020Dynamic Curriculum Learning for Low-Resource Neural Machine Translationcited by 11
  29. 2021Competence-based Curriculum Learning for Multilingual Machine Translationcited by 8
  30. 2018Reinforced Co-Trainingcited by 7
  31. 2024Can the Variation of Model Weights be used as a Criterion for Self-Paced Multilingual NMT?cited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2020Learning to Multi-Task Learn for Better Neural Machine Translation
  2. 2026Mending the Holes: Mitigating Reward Hacking in Reinforcement Learning for Multilingual Translation
  3. 2026Reinforcement Learning Elicits Contextual Learning of Unseen Language Translation
  4. 2025Improving Low-Resource Translation with Dictionary-Guided Fine-Tuning and RL: A Spanish-to-Wayuunaiki Study
  5. 2026Improving Low-Resource Machine Translation via Round-Trip Reinforcement Learning
  6. 2026Backtranslation Augmented Direct Preference Optimization for Neural Machine Translation
  7. 2026Learning to Route Languages for Multilingual Policy Optimization
  8. 2026Translate-R1: Cost-Aware Translation Tool Use via Reinforcement Learning