mlsanityData to 14 Oct 2026

Can the Variation of Model Weights be used as a Criterion for Self-Paced Multilingual NMT?

arXiv 2410.04147 ↗First posted 5 Oct 2024cs.CLcited by 1

Not on the map. Why?

Abstract (arXiv)

Many-to-one neural machine translation systems improve over one-to-one systems when training data is scarce. In this paper, we design and test a novel algorithm for selecting the language of minibatches when training such systems. The algorithm changes the language of the minibatch when the weights of the model do not evolve significantly, as measured by the smoothed KL divergence between all layers of the Transformer network. This algorithm outperforms the use of alternating monolingual batches, but not the use of shuffled batches, in terms of translation quality (measured with BLEU and COMET) and convergence speed.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 23 of its 30 references that are in the map. First posted on arXiv 5 Oct 2024.

Builds on

23 of 30 references.

References · 23

  1. 2017Attention Is All You Needcited by 60,361
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,448
  4. 2019Unsupervised Cross-lingual Representation Learning at Scalecited by 3,935
  5. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  6. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,799
  7. 2020COMET: A Neural Framework for MT Evaluationcited by 1,033
  8. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 469
  9. 2019Massively Multilingual Neural Machine Translation in the Wild: Findings and Challengescited by 257
  10. 2017OpenNMT: Open-source Toolkit for Neural Machine Translationcited by 251
  11. 2019Massively Multilingual Neural Machine Translationcited by 246
  12. 2021R-Drop: Regularized Dropout for Neural Networkscited by 202
11 more
  1. 2018When and Why are Pre-trained Word Embeddings Useful for Neural Machine Translation?cited by 126
  2. 2016Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanismcited by 117
  3. 2018Rapid Adaptation of Neural Machine Translation to New Languagescited by 82
  4. 2020Balancing Training for Multilingual Neural Machine Translationcited by 72
  5. 2018Universal Neural Machine Translation for Extremely Low Resource Languagescited by 66
  6. 2020Norm-Based Curriculum Learning for Neural Machine Translationcited by 60
  7. 2020Self-Paced Learning for Neural Machine Translationcited by 27
  8. 2019Adaptive Scheduling for Multi-Task Learningcited by 24
  9. 2019Multilingual Neural Machine Translation With Soft Decoupled Encodingcited by 21
  10. 2021Uncertainty-Aware Balancing for Multilingual and Multi-Domain Neural Machine Translation Trainingcited by 19
  11. 2021Competence-based Curriculum Learning for Multilingual Machine Translationcited by 8

Built on by

Cited by year

Citing papers per year: 2024: 1.

1 paper.

  1. 2024Optimizing the Training Schedule of Multilingual NMT using Reinforcement Learningcited by 0