mlsanityData to 7 Oct 2026

Norm-Based Curriculum Learning for Neural Machine Translation

arXiv 2006.02014 ↗First posted 3 Jun 2020cs.CL cs.LGcited by 60

Not on the map. Why?

Abstract (arXiv)

A neural machine translation (NMT) system is expensive to train, especially with high-resource settings. As the NMT architectures become deeper and wider, this issue gets worse and worse. In this paper, we aim to improve the efficiency of training an NMT by introducing a novel norm-based curriculum learning method. We use the norm (aka length or module) of a word embedding as a measure of 1) the difficulty of the sentence, 2) the competence of the model, and 3) the weight of the sentence. The norm-based sentence difficulty takes the advantages of both linguistically motivated and model-based sentence difficulties. It is easy to determine and contains learning-dependent features. The norm-based model competence makes NMT learn the curriculum in a fully automated way, while the norm-based sentence weight further enhances the learning of the vector representation of the NMT. Experimental results for the WMT'14 English-German and WMT'17 Chinese-English translation tasks demonstrate that the proposed method outperforms strong baselines in terms of BLEU score (+1.17/+1.56) and training speedup (2.22x/3.33x).

Key facts. Cited by 60 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Jun 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 19; 2022: 17; 2023: 4; 2024: 14; 2025: 3; 2026: 3.

60 papers.

Sort:

By area:

  1. 2021Curriculum Learning: A Surveycited by 257
  2. 2022Competence-based Multimodal Curriculum Learning for Medical Report Generationcited by 74
  3. 2021Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Surveycited by 46
  4. 2021Selective Knowledge Distillation for Neural Machine Translationcited by 42
  5. 2021Preview, Attend and Review: Schema-Aware Curriculum Learning for Multi-Domain Dialog State Trackingcited by 35
  6. 2021Does the Order of Training Samples Matter? Improving Neural Data-to-Text Generation with Curriculum Learningcited by 34
  7. 2022A Simple Hash-Based Early Exiting Approach For Language Understanding and Generationcited by 32
  8. 2021Deep Spatially and Temporally Aware Similarity Computation for Road Network Constrained Trajectoriescited by 29
  9. 2021Meta-Curriculum Learning for Domain Adaptation in Neural Machine Translationcited by 28
  10. 2021Progressive Multi-Granularity Training for Non-Autoregressive Translationcited by 25
  11. 2021Hybrid Curriculum Learning for Emotion Recognition in Conversationcited by 25
  12. 2022An Imitation Learning Curriculum for Text Editing with Non-Autoregressive Modelscited by 20
38 more
  1. 2021Improving Neural Machine Translation by Bidirectional Trainingcited by 17
  2. 2022Wait-info Policy: Balancing Source and Target at Information Level for Simultaneous Machine Translationcited by 16
  3. 2024Boost Your Human Image Generation Model via Direct Preference Optimizationcited by 16
  4. 2021On the Complementarity between Pre-Training and Back-Translation for Neural Machine Translationcited by 14
  5. 2022Improving Simultaneous Machine Translation with Monolingual Datacited by 13
  6. 2022A Contrastive Cross-Channel Data Augmentation Framework for Aspect-based Sentiment Analysiscited by 12
  7. 2021The USYD-JD Speech Translation System for IWSLT 2021cited by 11
  8. 2022Data Selection Curriculum for Neural Machine Translationcited by 11
  9. 2024From Easy to Hard: Progressive Active Learning Framework for Infrared Small Target Detection with Single Point Supervisioncited by 10
  10. 2021Self-Guided Curriculum Learning for Neural Machine Translationcited by 9
  11. 2022Vega-MT: The JD Explore Academy Translation System for WMT22cited by 9
  12. 2021Density-Based Dynamic Curriculum Learning for Intent Detectioncited by 7
  13. 2021Difficulty-Aware Machine Translation Evaluationcited by 6
  14. 2021Exploiting Curriculum Learning in Unsupervised Neural Machine Translationcited by 6
  15. 2022Breaking the Representation Bottleneck of Chinese Characters: Neural Machine Translation with Stroke Sequence Modelingcited by 6
  16. 2023M2DF: Multi-grained Multi-curriculum Denoising Framework for Multimodal Aspect-based Sentiment Analysiscited by 6
  17. 2025Estimating Machine Translation Difficultycited by 6
  18. 2022Training Dynamics for Curriculum Learning: A Study on Monolingual and Cross-lingual NLUcited by 5
  19. 2024Re-Examine Distantly Supervised NER: A New Benchmark and a Simple Approachcited by 5
  20. 2024Look Hear: Gaze Prediction for Speech-directed Human Attentioncited by 5
  21. 2021Token-wise Curriculum Learning for Neural Machine Translationcited by 4
  22. 2022Improving Neural Machine Translation by Denoising Trainingcited by 4
  23. 2022ConsistTL: Modeling Consistency in Transfer Learning for Low-Resource Neural Machine Translationcited by 4
  24. 20243AM: An Ambiguity-Aware Multi-Modal Machine Translation Datasetcited by 4
  25. 2024Let's Be Self-generated via Step by Step: A Curriculum Learning Approach to Automated Reasoning with Large Language Modelscited by 4
  26. 2022Less Emphasis on Difficult Layer Regions: Curriculum Learning for Singularly Perturbed Convection-Diffusion-Reaction Problemscited by 3
  27. 2022Cross-Modal Similarity-Based Curriculum Learning for Image Captioningcited by 3
  28. 2024Loss-Aware Curriculum Learning for Chinese Grammatical Error Correctioncited by 3
  29. 2025Task-Informed Anti-Curriculum by Masking Improves Downstream Performance on Textcited by 3
  30. 2022Adaptive Multi-Corpora Language Model Training for Speech Recognitioncited by 2
  31. 2023Clustering Pseudo Language Family in Multilingual Translation Models with Fisher Information Matrixcited by 2
  32. 2024Efficient k-Nearest-Neighbor Machine Translation with Dynamic Retrievalcited by 2
  33. 2024Fixed and Adaptive Simultaneous Machine Translation Strategies Using Adapterscited by 2
  34. 2021Statistical Measures For Defining Curriculum Scoring Functioncited by 1
  35. 2022Curriculum optimization for low-resource speech recognitioncited by 1
  36. 2023ERNetCL: A novel emotion recognition network in textual conversation based on curriculum learning strategycited by 1
  37. 2023Curriculum-Enhanced Residual Soft An-Isotropic Normalization for Over-smoothness in Deep GNNscited by 1
  38. 2024Enhancing Low-Resource NMT with a Multilingual Encoder and Knowledge Distillation: A Case Studycited by 1

50 most cited of 60 All 60 (JSON)