# Reinforcement Learning based Curriculum Optimization for Neural Machine Translation

arXiv: https://arxiv.org/abs/1903.00041 · HTML: https://mlsanity.com/paper/1903.00041/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Feb 2019.

## Built on by (most cited first, 29 of 29)

- [Curriculum Learning: A Survey](https://mlsanity.com/paper/2101.10382/) (2021)
- [Guiding Large Language Models via Directional Stimulus Prompting](https://mlsanity.com/paper/2302.11520/) (2023)
- [Competence-based Multimodal Curriculum Learning for Medical Report Generation](https://mlsanity.com/paper/2206.14579/) (2022)
- [Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey](https://mlsanity.com/paper/2104.06951/) (2021)
- [An Imitation Learning Curriculum for Text Editing with Non-Autoregressive Models](https://mlsanity.com/paper/2203.09486/) (2022)
- [Make Every Example Count: On the Stability and Utility of Self-Influence for Learning from Noisy NLP Datasets](https://mlsanity.com/paper/2302.13959/) (2023)
- [Bandits Don't Follow Rules: Balancing Multi-Facet Machine Translation with Multi-Armed Bandits](https://mlsanity.com/paper/2110.06997/) (2021)
- [Data Selection Curriculum for Neural Machine Translation](https://mlsanity.com/paper/2203.13867/) (2022)
- [Adaptive Training Distributions with Scalable Online Bilevel Optimization](https://mlsanity.com/paper/2311.11973/) (2023)
- [On the Complementarity of Data Selection and Fine Tuning for Domain Adaptation](https://mlsanity.com/paper/2109.07591/) (2021)
- [Generative Entity Typing with Curriculum Learning](https://mlsanity.com/paper/2210.02914/) (2022)
- [Fixing MoE Over-Fitting on Low-Resource Languages in Multilingual Machine Translation](https://mlsanity.com/paper/2212.07571/) (2022)
- [A Psychology-based Unified Dynamic Framework for Curriculum Learning](https://mlsanity.com/paper/2408.05326/) (2024)
- [Training Dynamics for Curriculum Learning: A Study on Monolingual and Cross-lingual NLU](https://mlsanity.com/paper/2210.12499/) (2022)
- [Robust Positive-Unlabeled Learning via Noise Negative Sample Self-correction](https://mlsanity.com/paper/2308.00279/) (2023)
- [Token-wise Curriculum Learning for Neural Machine Translation](https://mlsanity.com/paper/2103.11088/) (2021)
- [In-sample Curriculum Learning by Sequence Completion for Natural Language Generation](https://mlsanity.com/paper/2211.11297/) (2022)
- [Mending the Holes: Mitigating Reward Hacking in Reinforcement Learning for Multilingual Translation](https://mlsanity.com/paper/2603.13045/) (2026)
- [Searching the Internet for Challenging Benchmarks at Scale](https://mlsanity.com/paper/2509.26619/) (2025)
- [Learning Policies for Multilingual Training of Neural Machine Translation Systems](https://mlsanity.com/paper/2103.06964/) (2021)
- [Learning Feature Weights using Reward Modeling for Denoising Parallel Corpora](https://mlsanity.com/paper/2103.06968/) (2021)
- [Curriculum optimization for low-resource speech recognition](https://mlsanity.com/paper/2202.08883/) (2022)
- [Efficient Distributed Framework for Collaborative Multi-Agent Reinforcement Learning](https://mlsanity.com/paper/2205.05248/) (2022)
- [Progressively Efficient Learning](https://mlsanity.com/paper/2310.13004/) (2023)
- [LexMatcher: Dictionary-centric Data Collection for LLM-based Machine Translation](https://mlsanity.com/paper/2406.01441/) (2024)
- [Adaptively Point-weighting Curriculum Learning](https://mlsanity.com/paper/2505.01665/) (2025)
- [Epi-Curriculum: Episodic Curriculum Learning for Low-Resource Domain Adaptation in Neural Machine Translation](https://mlsanity.com/paper/2309.02640/) (2023)
- [Optimizing the Training Schedule of Multilingual NMT using Reinforcement Learning](https://mlsanity.com/paper/2410.06118/) (2024)
- [Disentangling Curriculum Learning in NLP: Towards a Unifying Taxonomy](https://mlsanity.com/paper/2607.18984/) (2026)

## Abstract (arXiv)

We consider the problem of making efficient use of heterogeneous training data in neural machine translation (NMT). Specifically, given a training dataset with a sentence-level feature such as noise, we seek an optimal curriculum, or order for presenting examples to the system during training. Our curriculum framework allows examples to appear an arbitrary number of times, and thus generalizes data weighting, filtering, and fine-tuning schemes. Rather than relying on prior knowledge to design a curriculum, we use reinforcement learning to learn one automatically, jointly with the NMT system, in the course of a single training run. We show that this approach can beat uniform and filtering baselines on Paracrawl and WMT English-to-French datasets by up to +3.4 BLEU, and match the performance of a hand-designed, state-of-the-art curriculum.