# Self-Paced Learning for Neural Machine Translation

arXiv: https://arxiv.org/abs/2010.04505 · HTML: https://mlsanity.com/paper/2010.04505/

## Key facts

- Cited by 27 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Oct 2020.

## Built on by (most cited first, 27 of 27)

- [Efficient Methods for Natural Language Processing: A Survey](https://mlsanity.com/paper/2209.00099/) (2022)
- [The Efficiency Spectrum of Large Language Models: An Algorithmic Survey](https://mlsanity.com/paper/2312.00678/) (2023)
- [NLU on Data Diets: Dynamic Data Subset Selection for NLP Classification Tasks](https://mlsanity.com/paper/2306.03208/) (2023)
- [Bridging Subword Gaps in Pretrain-Finetune Paradigm for Natural Language Generation](https://mlsanity.com/paper/2106.06125/) (2021)
- [Frequency-Aware Contrastive Learning for Neural Machine Translation](https://mlsanity.com/paper/2112.14484/) (2021)
- [Data Selection Curriculum for Neural Machine Translation](https://mlsanity.com/paper/2203.13867/) (2022)
- [Poor Man's Quality Estimation: Predicting Reference-Based MT Metrics Without the Reference](https://mlsanity.com/paper/2301.09008/) (2023)
- [Towards User-Driven Neural Machine Translation](https://mlsanity.com/paper/2106.06200/) (2021)
- [Confidence-Aware Scheduled Sampling for Neural Machine Translation](https://mlsanity.com/paper/2107.10427/) (2021)
- [Alibaba-Translate China's Submission for WMT 2022 Metrics Shared Task](https://mlsanity.com/paper/2210.09683/) (2022)
- [Exploiting Curriculum Learning in Unsupervised Neural Machine Translation](https://mlsanity.com/paper/2109.11177/) (2021)
- [Conditional Bilingual Mutual Information Based Adaptive Training for Neural Machine Translation](https://mlsanity.com/paper/2203.02951/) (2022)
- [A Psychology-based Unified Dynamic Framework for Curriculum Learning](https://mlsanity.com/paper/2408.05326/) (2024)
- [Bilingual Mutual Information Based Adaptive Training for Neural Machine Translation](https://mlsanity.com/paper/2105.12523/) (2021)
- [Training Dynamics for Curriculum Learning: A Study on Monolingual and Cross-lingual NLU](https://mlsanity.com/paper/2210.12499/) (2022)
- [Self-Modifying State Modeling for Simultaneous Machine Translation](https://mlsanity.com/paper/2406.02237/) (2024)
- [Alibaba-Translate China's Submission for WMT 2022 Quality Estimation Shared Task](https://mlsanity.com/paper/2210.10049/) (2022)
- [A Multi-Modal Knowledge-Enhanced Framework for Vessel Trajectory Prediction](https://mlsanity.com/paper/2503.21834/) (2025)
- [Competency-Aware Neural Machine Translation: Can Machine Translation Know its Own Translation Quality?](https://mlsanity.com/paper/2211.13865/) (2022)
- [SAS: Self-Augmentation Strategy for Language Model Pre-training](https://mlsanity.com/paper/2106.07176/) (2021)
- [CBSiMT: Mitigating Hallucination in Simultaneous Machine Translation with Weighted Prefix-to-Prefix Training](https://mlsanity.com/paper/2311.03672/) (2023)
- [An Effective Incorporating Heterogeneous Knowledge Curriculum Learning for Sequence Labeling](https://mlsanity.com/paper/2402.13534/) (2024)
- [Can the Variation of Model Weights be used as a Criterion for Self-Paced Multilingual NMT?](https://mlsanity.com/paper/2410.04147/) (2024)
- [Balanced Self-Paced Learning for AUC Maximization](https://mlsanity.com/paper/2207.03650/) (2022)
- [Optimizing the Training Schedule of Multilingual NMT using Reinforcement Learning](https://mlsanity.com/paper/2410.06118/) (2024)
- [Testing the Assumptions of Active Learning for Translation Tasks with Few Samples](https://mlsanity.com/paper/2604.08977/) (2026)
- [Disentangling Curriculum Learning in NLP: Towards a Unifying Taxonomy](https://mlsanity.com/paper/2607.18984/) (2026)

## Abstract (arXiv)

Recent studies have proven that the training of neural machine translation (NMT) can be facilitated by mimicking the learning process of humans. Nevertheless, achievements of such kind of curriculum learning rely on the quality of artificial schedule drawn up with the handcrafted features, e.g. sentence length or word rarity. We ameliorate this procedure with a more flexible manner by proposing self-paced learning, where NMT model is allowed to 1) automatically quantify the learning confidence over training examples; and 2) flexibly govern its learning via regulating the loss in each iteration step. Experimental results over multiple translation tasks demonstrate that the proposed model yields better performance than strong baselines and those models trained with human-designed curricula on both translation quality and convergence speed.