Low-Resource Machine Translation through the Lens of Personalized Federated Learning
Not on the map. Why?
Abstract (arXiv)
We present a new approach called MeritOpt based on the Personalized Federated Learning algorithm MeritFed that can be applied to Natural Language Tasks with heterogeneous data. We evaluate it on the Low-Resource Machine Translation task, using the datasets of South East Asian and Finno-Ugric languages. In addition to its effectiveness, MeritOpt is also highly interpretable, as it can be applied to track the impact of each language used for training. Our analysis reveals that target dataset size affects weight distribution across auxiliary languages, that unrelated languages do not interfere with the training, and auxiliary optimizer parameters have minimal impact. Our approach is easy to apply with a few lines of code, and we provide scripts for reproducing the experiments at https://github.com/VityaVitalich/MeritOpt.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 26 of its 53 references that are in the map. First posted on arXiv 18 Jun 2024.
Builds on
26 of 53 references.
References · 26
- 2014Adam: A Method for Stochastic Optimizationcited by 36,960
- 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
- 2016Communication-Efficient Learning of Deep Networks from Decentralized Datacited by 7,592
- 2019Advances and Open Problems in Federated Learningcited by 3,021
- 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
- 2016Federated Learning: Strategies for Improving Communication Efficiencycited by 1,461
- 2016On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minimacited by 1,337
- 2020Beyond English-Centric Multilingual Machine Translationcited by 559
- 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 556
- 2021Exploiting Shared Representations for Personalized Federated Learningcited by 537
- 2020Personalized Federated Learning: A Meta-Learning Approachcited by 309
- 2020A Survey on Recent Approaches for Natural Language Processing in Low-Resource Scenarioscited by 170
14 more
- 2020Survey of Personalization Techniques for Federated Learningcited by 144
- 2020Lower Bounds and Optimal Algorithms for Personalized Federated Learningcited by 122
- 2023Glot500: Scaling Multilingual Corpora and Language Models to 500 Languagescited by 116
- 2021Neural Machine Translation for Low-Resource Languages: A Surveycited by 108
- 2021Survey of Low-Resource Machine Translationcited by 92
- 2020Balancing Training for Multilingual Neural Machine Translationcited by 71
- 2020Fast-Convergent Federated Learning with Adaptive Weightingcited by 67
- 2021DeltaLM: Encoder-Decoder Pre-training for Language Generation and Translation by Augmenting Pretrained Multilingual Encoderscited by 50
- 2021Multilingual Machine Translation Systems from Microsoft for WMT21 Shared Taskcited by 34
- 2021A Survey on Low-Resource Neural Machine Translationcited by 16
- 2021Back-translation for Large-Scale Multilingual Machine Translationcited by 11
- 2020Word Sense Disambiguation for 158 Languages using Word Embeddings Onlycited by 4
- 2024Methods of Nonconvex Optimizationcited by 3
- 2021Data Processing Matters: SRPH-Konvergen AI's Machine Translation System for WMT'21cited by 2
Built on by
Citing papers per year: 2025: 1.
1 paper.