mlsanityData to 7 Oct 2026

Data Selection Curriculum for Neural Machine Translation

arXiv 2203.13867 ↗First posted 25 Mar 2022cs.CL cs.LGcited by 11

Not on the map. Why?

Abstract (arXiv)

Neural Machine Translation (NMT) models are typically trained on heterogeneous data that are concatenated and randomly shuffled. However, not all of the training data are equally useful to the model. Curriculum training aims to present the data to the NMT models in a meaningful order. In this work, we introduce a two-stage curriculum training framework for NMT where we fine-tune a base NMT model on subsets of data, selected by both deterministic scoring using pre-trained methods and online scoring that considers prediction scores of the emerging NMT model. Through comprehensive experiments on six language pairs comprising low- and high-resource languages from WMT'21, we have shown that our curriculum strategies consistently demonstrate better quality (up to +2.2 BLEU improvement) and faster convergence (approximately 50% fewer updates).

Key facts. Cited by 11 papers in the map (data through 7 Oct 2026). Builds on 30 of its 47 references that are in the map. First posted on arXiv 25 Mar 2022.

Builds on

30 of 47 references.

References · 30

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  3. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  4. 2017Word Translation Without Parallel Datacited by 480
  5. 2019On The Power of Curriculum Learning in Training Deep Networkscited by 285
  6. 2019Competence-based Curriculum Learning for Neural Machine Translationcited by 201
  7. 2016Transfer Learning for Low-Resource Neural Machine Translationcited by 188
  8. 2018Achieving Human Parity on Automatic Chinese to English News Translationcited by 119
  9. 2018Margin-based Parallel Corpus Mining with Multilingual Sentence Embeddingscited by 83
  10. 2018On the Impact of Various Types of Noise on Neural Machine Translationcited by 76
  11. 2017Curriculum Learning and Minibatch Bucketing in Neural Machine Translationcited by 63
  12. 2020Norm-Based Curriculum Learning for Neural Machine Translationcited by 60
18 more
  1. 2019Simple and Effective Curriculum Pointer-Generator Networks for Reading Comprehension over Long Narrativescited by 59
  2. 2018An Empirical Exploration of Curriculum Learning for Neural Machine Translationcited by 58
  3. 2016Visualizing and Understanding Curriculum Learning for Long Short-Term Memory Networkscited by 44
  4. 2018Denoising Neural Machine Translation Training with Trusted Data and Online Data Selectioncited by 43
  5. 2018Dual Conditional Cross-Entropy Filtering of Noisy Parallel Corporacited by 40
  6. 2017Dynamic Data Selection for Neural Machine Translationcited by 33
  7. 2021Curriculum learning for language modelingcited by 32
  8. 2019Reinforcement Learning based Curriculum Optimization for Neural Machine Translationcited by 29
  9. 2016Fast Domain Adaptation for Neural Machine Translationcited by 28
  10. 2021Meta-Curriculum Learning for Domain Adaptation in Neural Machine Translationcited by 28
  11. 2020Self-Paced Learning for Neural Machine Translationcited by 27
  12. 2019Low-Resource Corpus Filtering using Multilingual Sentence Embeddingscited by 25
  13. 2016Learning the Curriculum with Bayesian Optimization for Task-Specific Word Representation Learningcited by 22
  14. 2020Reinforced Curriculum Learning on Pre-trained Neural Machine Translation Modelscited by 20
  15. 2020Data Rejuvenation: Exploiting Inactive Training Examples for Neural Machine Translationcited by 17
  16. 2020Dynamic Curriculum Learning for Low-Resource Neural Machine Translationcited by 11
  17. 2021Self-Guided Curriculum Learning for Neural Machine Translationcited by 9
  18. 2021Gradient-guided Loss Masking for Neural Machine Translationcited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 2; 2024: 5; 2025: 2; 2026: 1.

11 papers.

  1. 2022On the Domain Adaptation and Generalization of Pretrained Language Models: A Surveycited by 22
  2. 2023Error Norm Truncation: Robust Training in the Presence of Data Noise for Text Generation Modelscited by 6
  3. 2024A Psychology-based Unified Dynamic Framework for Curriculum Learningcited by 6
  4. 2024Let's Be Self-generated via Step by Step: A Curriculum Learning Approach to Automated Reasoning with Large Language Modelscited by 4
  5. 2023Ling-CL: Understanding NLP Models through Linguistic Curriculacited by 3
  6. 2024Towards Inducing Long-Context Abilities in Multilingual Neural Machine Translation Modelscited by 3
  7. 2025RIVAL: Reinforcement Learning with Iterative and Adversarial Optimization for Machine Translationcited by 3
  8. 2025Beyond Shallow Heuristics: Leveraging Human Intuition for Curriculum Learningcited by 2
  9. 2024An Effective Incorporating Heterogeneous Knowledge Curriculum Learning for Sequence Labelingcited by 1
  10. 2024LexMatcher: Dictionary-centric Data Collection for LLM-based Machine Translationcited by 1
  11. 2026Testing the Assumptions of Active Learning for Translation Tasks with Few Samplescited by 0