mlsanityData to 7 Oct 2026

Balancing Training for Multilingual Neural Machine Translation

arXiv 2004.06748 ↗First posted 14 Apr 2020cs.CLcited by 71

Not on the map. Why?

Abstract (arXiv)

When training multilingual machine translation (MT) models that can translate to/from multiple languages, we are faced with imbalanced training sets: some languages have much more training data than others. Standard practice is to up-sample less resourced languages to increase representation, and the degree of up-sampling has a large effect on the overall performance. In this paper, we propose a method that instead automatically learns how to weight training data through a data scorer that is optimized to maximize performance on all test languages. Experiments on two sets of languages under both one-to-many and many-to-one MT settings show our method not only consistently outperforms heuristic baselines in terms of average performance, but also offers flexible control over the performance of which languages are optimized.

Key facts. Cited by 71 papers in the map (data through 7 Oct 2026). First posted on arXiv 14 Apr 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 22; 2022: 12; 2023: 14; 2024: 11; 2025: 6; 2026: 6.

71 papers.

Sort:

By area:

  1. 2024Aya Model: An Instruction Finetuned Open-Access Multilingual Language Modelcited by 292
  2. 2023A Pretrainer's Guide to Training Data: Measuring the Effects of Data Age, Domain Coverage, Quality, & Toxicitycited by 180
  3. 2023UniMax: Fairer and more Effective Language Sampling for Large-Scale Multilingual Pretrainingcited by 147
  4. 2021Neural Machine Translation for Low-Resource Languages: A Surveycited by 108
  5. 2024What is Wrong with Perplexity for Long-context Language Modeling?cited by 68
  6. 2021Scaling End-to-End Models for Large-Scale Multilingual ASRcited by 61
  7. 2022Uni-Perceiver-MoE: Learning Sparse Generalist Models with Conditional MoEscited by 58
  8. 2021Multi-Objective Meta Learningcited by 39
  9. 2021Turning Tables: Generating Examples from Semi-structured Tables for Endowing Language Models with Reasoning Skillscited by 39
  10. 2021Multilingual Machine Translation Systems from Microsoft for WMT21 Shared Taskcited by 34
  11. 2023Measuring The Impact Of Programming Language Distributioncited by 32
  12. 2021Many-to-English Machine Translation Tools, Data, and Pretrained Modelscited by 31
38 more
  1. 2021Improving Multilingual Translation by Representation and Gradient Regularizationcited by 31
  2. 2023Scaling Laws for Multilingual Neural Machine Translationcited by 27
  3. 2022Meta Learning for Natural Language Processing: A Surveycited by 26
  4. 2022Causes and Cures for Interference in Multilingual Translationcited by 23
  5. 2021Uncertainty-Aware Balancing for Multilingual and Multi-Domain Neural Machine Translation Trainingcited by 19
  6. 2021Distributionally Robust Multilingual Machine Translationcited by 18
  7. 2021Tricks for Training Sparse Translation Modelscited by 17
  8. 2023RECKONING: Reasoning through Dynamic Knowledge Encodingcited by 17
  9. 2021Meta Back-translationcited by 16
  10. 2021A Survey on Low-Resource Neural Machine Translationcited by 16
  11. 2021Sequential Reptile: Inter-Task Gradient Alignment for Multilingual Learningcited by 16
  12. 2021Bandits Don't Follow Rules: Balancing Multi-Facet Machine Translation with Multi-Armed Banditscited by 16
  13. 2021Robust Optimization for Multilingual Translation with Imbalanced Datacited by 15
  14. 2024Mixture-of-Skills: Learning to Optimize Data Usage for Fine-Tuning Large Language Modelscited by 15
  15. 2021Meta-Learning for Effective Multi-task and Multilingual Modellingcited by 13
  16. 2022Tencent's Multilingual Machine Translation System for WMT22 Large-Scale African Languagescited by 13
  17. 2023Learning Language-Specific Layers for Multilingual Machine Translationcited by 13
  18. 2021Efficient Weight factorization for Multilingual Speech Recognitioncited by 12
  19. 2022Multilingual Neural Machine Translation with Deep Encoder and Multiple Shallow Decoderscited by 12
  20. 2021Minimax and Neyman-Pearson Meta-Learning for Outlier Languagescited by 10
  21. 2023Towards Higher Pareto Frontier in Multilingual Machine Translationcited by 9
  22. 2021Gradient-guided Loss Masking for Neural Machine Translationcited by 8
  23. 2021Adaptive Sparse Transformer for Multilingual Translationcited by 8
  24. 2021Competence-based Curriculum Learning for Multilingual Machine Translationcited by 8
  25. 2023Bridging the Language Gap: Dynamic Learning Strategies for Improving Multilingual Performance in LLMscited by 8
  26. 2023On the Pareto Front of Multilingual Neural Machine Translationcited by 7
  27. 2023Neural Machine Translation for the Indigenous Languages of the Americas: An Introductioncited by 7
  28. 2023Order Matters in the Presence of Dataset Imbalance for Multilingual Learningcited by 7
  29. 2024To Label or Not to Label: Hybrid Active Learning for Neural Machine Translationcited by 7
  30. 2022Por Qu'e N~ao Utiliser Alla Spraak? Mixed Training with Gradient Optimization in Few-Shot Cross-Lingual Transfercited by 6
  31. 2022You Can Have Your Data and Balance It Too: Towards Balanced and Efficient Multilingual Modelscited by 6
  32. 2023LIMIT: Language Identification, Misidentification, and Translation using Hierarchical Models in 350+ Languagescited by 6
  33. 2023Error Norm Truncation: Robust Training in the Presence of Data Noise for Text Generation Modelscited by 6
  34. 2025DynamixSFT: Dynamic Mixture Optimization of Instruction Tuning Collectionscited by 6
  35. 2022Forging Multiple Training Objectives for Pre-trained Language Models via Meta-Learningcited by 5
  36. 2024DRPruning: Efficient Large Language Model Pruning through Distributionally Robust Optimizationcited by 5
  37. 2024Upsample or Upweight? Balanced Training on Heavily Imbalanced Datasetscited by 4
  38. 2026Rethinking Personalization in Large Language Models at the Token Levelcited by 3

50 most cited of 71 All 71 (JSON)