mlsanityData to 7 Oct 2026

Competence-based Curriculum Learning for Neural Machine Translation

arXiv 1903.09848 ↗First posted 23 Mar 2019cs.CL cs.LG stat.MLcited by 201

Not on the map. Why?

Abstract (arXiv)

Current state-of-the-art NMT systems use large neural networks that are not only slow to train, but also often require many heuristics and optimization tricks, such as specialized learning rate schedules and large batch sizes. This is undesirable as it requires extensive hyperparameter tuning. In this paper, we propose a curriculum learning framework for NMT that reduces training time, reduces the need for specialized heuristics or large batch sizes, and results in overall better performance. Our framework consists of a principled way of deciding which training samples are shown to the model at different times during training, based on the estimated difficulty of a sample and the current competence of the model. Filtering training samples in this manner prevents the model from getting stuck in bad local optima, making it converge faster and reach a better solution than the common approach of uniformly sampling training examples. Furthermore, the proposed method can be easily applied to existing NMT models by simply modifying their input data pipelines. We show that our framework can help improve the training time and the performance of both recurrent neural network models and Transformers, achieving up to a 70% decrease in training time, while at the same time obtaining accuracy improvements of up to 2.2 BLEU.

Key facts. Cited by 201 papers in the map (data through 7 Oct 2026). First posted on arXiv 23 Mar 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 25; 2022: 36; 2023: 38; 2024: 39; 2025: 32; 2026: 31.

201 papers.

Sort:

By area:

  1. 2021Curriculum Learning: A Surveycited by 257
  2. 2025A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligencecited by 155
  3. 2022Curriculum Temperature for Knowledge Distillationcited by 110
  4. 2025Online Difficulty Filtering for Reasoning Oriented Reinforcement Learningcited by 100
  5. 2022Efficient Methods for Natural Language Processing: A Surveycited by 78
  6. 2022Competence-based Multimodal Curriculum Learning for Medical Report Generationcited by 74
  7. 2023Generalization on the Unseen, Logic Reasoning and Degree Curriculumcited by 73
  8. 2023A-JEPA: Joint-Embedding Predictive Architecture Can Listencited by 50
  9. 2021Selective Knowledge Distillation for Neural Machine Translationcited by 42
  10. 2022EfficientTrain: Exploring Generalized Curriculum Learning for Training Visual Backbonescited by 42
  11. 2022A Survey of Active Learning for Natural Language Processingcited by 40
  12. 2024Curry-DPO: Enhancing Alignment using Curriculum Learning & Ranked Preferencescited by 39
38 more
  1. 2025Principled Data Selection for Alignment: The Hidden Risks of Difficult Examplescited by 37
  2. 2021The Stability-Efficiency Dilemma: Investigating Sequence Length Warmup for Training GPT Modelscited by 36
  3. 2021Does the Order of Training Samples Matter? Improving Neural Data-to-Text Generation with Curriculum Learningcited by 34
  4. 2021Rejuvenating Low-Frequency Words: Making the Most of Parallel Data in Non-Autoregressive Translationcited by 34
  5. 2021Self-Paced Contrastive Learning for Semi-supervised Medical Image Segmentation with Meta-labelscited by 32
  6. 2021Curriculum learning for language modelingcited by 32
  7. 2025SurgVLM: A Large Vision-Language Model and Systematic Evaluation Benchmark for Surgical Intelligencecited by 30
  8. 2021Meta-Curriculum Learning for Domain Adaptation in Neural Machine Translationcited by 28
  9. 2024Strategic Data Ordering: Enhancing Large Language Model Performance through Curriculum Learningcited by 28
  10. 2021Bridging Pre-trained Models and Downstream Tasks for Source Code Understandingcited by 27
  11. 2022Learning to Solve Travelling Salesman Problem with Hardness-adaptive Curriculumcited by 27
  12. 2023ASP: Learn a Universal Neural Solver!cited by 27
  13. 2021Progressive Multi-Granularity Training for Non-Autoregressive Translationcited by 25
  14. 2022A Model-Agnostic Data Manipulation Method for Persona-based Dialogue Generationcited by 25
  15. 2022CLNode: Curriculum Learning for Node Classificationcited by 24
  16. 2023Provable Advantage of Curriculum Learning on Parity Targets with Mixed Inputscited by 22
  17. 2021Gradual Domain Adaptation in the Wild:When Intermediate Distributions are Absentcited by 21
  18. 2023CL-MAE: Curriculum-Learned Masked Autoencoderscited by 21
  19. 2022An Imitation Learning Curriculum for Text Editing with Non-Autoregressive Modelscited by 20
  20. 2023CLIMB: Curriculum Learning for Infant-inspired Model Buildingcited by 20
  21. 2023NLU on Data Diets: Dynamic Data Subset Selection for NLP Classification Taskscited by 19
  22. 2021Curriculum Learning for Vision-and-Language Navigationcited by 18
  23. 2023Make Every Example Count: On the Stability and Utility of Self-Influence for Learning from Noisy NLP Datasetscited by 18
  24. 2021Self-Training Sampling with Monolingual Data Uncertainty for Neural Machine Translationcited by 17
  25. 2021Bandits Don't Follow Rules: Balancing Multi-Facet Machine Translation with Multi-Armed Banditscited by 16
  26. 2024EfficientTrain++: Generalized Curriculum Learning for Efficient Visual Backbone Trainingcited by 16
  27. 2023Curriculum Graph Machine Learning: A Surveycited by 15
  28. 2022PCC: Paraphrasing with Bottom-k Sampling and Cyclic Learning for Curriculum Data Augmentationcited by 14
  29. 2024Towards Faster Training of Diffusion Models: An Inspiration of A Consistency Phenomenoncited by 14
  30. 2022Improving Simultaneous Machine Translation with Monolingual Datacited by 13
  31. 2022DeepSpeed Data Efficiency: Improving Deep Learning Model Quality and Training Efficiency via Efficient Data Sampling and Routingcited by 13
  32. 2023ViStruct: Visual Structural Knowledge Extraction via Curriculum Guided Code-Vision Representationcited by 13
  33. 2024Intent-based Prompt Calibration: Enhancing prompt optimization with synthetic boundary casescited by 13
  34. 2024CommonIT: Commonality-Aware Instruction Tuning for Large Language Models via Data Partitionscited by 13
  35. 2022Data Selection Curriculum for Neural Machine Translationcited by 11
  36. 2021Semi-Autoregressive Image Captioningcited by 10
  37. 2021Hierarchical Curriculum Learning for AMR Parsingcited by 10
  38. 2023Advising OpenMP Parallelization via a Graph-Based Approach with Transformerscited by 10

50 most cited of 201 All 201 (JSON)