# An Empirical Exploration of Curriculum Learning for Neural Machine Translation

arXiv: https://arxiv.org/abs/1811.00739 · HTML: https://mlsanity.com/paper/1811.00739/

## Key facts

- Cited by 58 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 2 Nov 2018.

## Built on by (most cited first, 30 of 58)

- [Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models](https://mlsanity.com/paper/2401.01335/) (2024)
- [Curriculum Learning: A Survey](https://mlsanity.com/paper/2101.10382/) (2021)
- [Competence-based Multimodal Curriculum Learning for Medical Report Generation](https://mlsanity.com/paper/2206.14579/) (2022)
- [Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey](https://mlsanity.com/paper/2104.06951/) (2021)
- [EfficientTrain: Exploring Generalized Curriculum Learning for Training Visual Backbones](https://mlsanity.com/paper/2211.09703/) (2022)
- [The Stability-Efficiency Dilemma: Investigating Sequence Length Warmup for Training GPT Models](https://mlsanity.com/paper/2108.06084/) (2021)
- [Does the Order of Training Samples Matter? Improving Neural Data-to-Text Generation with Curriculum Learning](https://mlsanity.com/paper/2102.03554/) (2021)
- [Meta-Curriculum Learning for Domain Adaptation in Neural Machine Translation](https://mlsanity.com/paper/2103.02262/) (2021)
- [Bridging Pre-trained Models and Downstream Tasks for Source Code Understanding](https://mlsanity.com/paper/2112.02268/) (2021)
- [Beyond Random Sampling: Efficient Language Model Pretraining via Curriculum Learning](https://mlsanity.com/paper/2506.11300/) (2025)
- [Provable Advantage of Curriculum Learning on Parity Targets with Mixed Inputs](https://mlsanity.com/paper/2306.16921/) (2023)
- [Gradual Domain Adaptation in the Wild:When Intermediate Distributions are Absent](https://mlsanity.com/paper/2106.06080/) (2021)
- [An Imitation Learning Curriculum for Text Editing with Non-Autoregressive Models](https://mlsanity.com/paper/2203.09486/) (2022)
- [EfficientTrain++: Generalized Curriculum Learning for Efficient Visual Backbone Training](https://mlsanity.com/paper/2405.08768/) (2024)
- [Bandits Don't Follow Rules: Balancing Multi-Facet Machine Translation with Multi-Armed Bandits](https://mlsanity.com/paper/2110.06997/) (2021)
- [A Systematic Review of Deep Learning-based Research on Radiology Report Generation](https://mlsanity.com/paper/2311.14199/) (2023)
- [DeepSpeed Data Efficiency: Improving Deep Learning Model Quality and Training Efficiency via Efficient Data Sampling and Routing](https://mlsanity.com/paper/2212.03597/) (2022)
- [Efficient Pre-training of Masked Language Model via Concept-based Curriculum Masking](https://mlsanity.com/paper/2212.07617/) (2022)
- [Curricular Object Manipulation in LiDAR-based Object Detection](https://mlsanity.com/paper/2304.04248/) (2023)
- [Data Selection Curriculum for Neural Machine Translation](https://mlsanity.com/paper/2203.13867/) (2022)
- [Self-Guided Curriculum Learning for Neural Machine Translation](https://mlsanity.com/paper/2105.04475/) (2021)
- [When Do Curricula Work in Federated Learning?](https://mlsanity.com/paper/2212.12712/) (2022)
- [Self-paced Multi-grained Cross-modal Interaction Modeling for Referring Expression Comprehension](https://mlsanity.com/paper/2204.09957/) (2022)
- [Fixing MoE Over-Fitting on Low-Resource Languages in Multilingual Machine Translation](https://mlsanity.com/paper/2212.07571/) (2022)
- [GaitMPL: Gait Recognition with Memory-Augmented Progressive Learning](https://mlsanity.com/paper/2306.04650/) (2023)
- [Too Much Information: Keeping Training Simple for BabyLMs](https://mlsanity.com/paper/2311.01955/) (2023)
- [Exploiting Curriculum Learning in Unsupervised Neural Machine Translation](https://mlsanity.com/paper/2109.11177/) (2021)
- [HiCuLR: Hierarchical Curriculum Learning for Rhetorical Role Labeling of Legal Documents](https://mlsanity.com/paper/2409.18647/) (2024)
- [Estimating Machine Translation Difficulty](https://mlsanity.com/paper/2508.10175/) (2025)
- [From Easy to Hard: A Dual Curriculum Learning Framework for Context-Aware Document Ranking](https://mlsanity.com/paper/2208.10226/) (2022)

## Abstract (arXiv)

Machine translation systems based on deep neural networks are expensive to train. Curriculum learning aims to address this issue by choosing the order in which samples are presented during training to help train better models faster. We adopt a probabilistic view of curriculum learning, which lets us flexibly evaluate the impact of curricula design, and perform an extensive exploration on a German-English translation task. Results show that it is possible to improve convergence time at no loss in translation quality. However, results are highly sensitive to the choice of sample difficulty criteria, curriculum schedule and other hyperparameters.