Extending Multilingual Machine Translation through Imitation Learning
Not on the map. Why?
Abstract (arXiv)
Despite the growing variety of languages supported by existing multilingual neural machine translation (MNMT) models, most of the world's languages are still being left behind. We aim to extend large-scale MNMT models to incorporate a new language, enabling translations between this new language and all previously supported languages, even in the challenging scenario where only a parallel corpus between the new language and English is available. Previous methods, such as continued training on parallel data including the new language, often suffer from catastrophic forgetting, which degrades performance on other languages. We propose a novel approach Imit-MNMT which treats this task as an imitation learning problem, a technique widely used in computer vision but less explored in natural language processing. Specifically, we leverage an expert model to generate pseudo-parallel corpora between the new language and the existing languages. We then introduce a data distribution imitation strategy using language-specific weighting, alongside a translation behavior imitation mechanism. Extensive experiments show that our approach significantly improves translation performance between the new and existing languages while mitigating catastrophic forgetting.
Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 43 of its 69 references that are in the map. First posted on arXiv 14 Nov 2023.
Builds on
43 of 69 references.
References · 43
- 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
- 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
- 2020Knowledge Distillation: A Surveycited by 1,578
- 2016Generative Adversarial Imitation Learningcited by 1,576
- 2022No Language Left Behind: Scaling Human-Centered Machine Translationcited by 1,290
- 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,028
- 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784
- 2019On the Cross-lingual Transferability of Monolingual Representationscited by 575
- 2020Beyond English-Centric Multilingual Machine Translationcited by 559
- 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 556
- 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 467
- 2020Improving Massively Multilingual Neural Machine Translation and Zero-Shot Translationcited by 277
31 more
- 2017Unsupervised Machine Translation Using Monolingual Corpora Onlycited by 272
- 2017Beam Search Strategies for Neural Machine Translationcited by 232
- 2021DexMV: Imitation Learning for Dexterous Manipulation from Human Videoscited by 194
- 2018Fast Lexically Constrained Decoding with Dynamic Beam Allocation for Neural Machine Translationcited by 181
- 2021To Ship or Not to Ship: An Extensive Evaluation of Automatic Metrics for Machine Translationcited by 161
- 2020UNKs Everywhere: Adapting Multilingual Language Models to New Scriptscited by 113
- 2020Extending Multilingual BERT to Low-Resource Languagescited by 88
- 2020Autoregressive Knowledge Distillation through Imitation Learningcited by 68
- 2021How to Adapt Your Pretrained Multilingual Model to 1600 Languagescited by 54
- 2021Learning Language Specific Sub-network for Multilingual Machine Translationcited by 48
- 2019Imitation Learning for Non-Autoregressive Neural Machine Translationcited by 36
- 2021Towards Continual Learning for Multilingual Machine Translation via Vocabulary Substitutioncited by 31
- 2022Mini-Model Adaptation: Efficiently Extending Pretrained Models to New Languages via Aligned Shallow Trainingcited by 24
- 2020An Imitation Game for Learning Semantic Parsers from User Interactioncited by 23
- 2018Transfer Learning in Multilingual Neural Machine Translation with Dynamic Vocabularycited by 18
- 2021Adapting High-resource NMT Models to Translate Low-resource Related Languages without Parallel Datacited by 17
- 2021On the Copying Behaviors of Pre-Training for Neural Machine Translationcited by 15
- 2022Language-Family Adapters for Low-Resource Multilingual Neural Machine Translationcited by 14
- 2023Learning Language-Specific Layers for Multilingual Machine Translationcited by 13
- 2021Continual Learning in Multilingual NMT via Language-Specific Embeddingscited by 12
- 2021Improving Both Domain Robustness and Domain Adaptability in Machine Translationcited by 11
- 2018Bi-Directional Neural Machine Translation with Synthetic Parallel Datacited by 10
- 2022Disentangling Uncertainty in Machine Translation Evaluationcited by 9
- 2022Continual Learning of Neural Machine Translation within Low Forgetting Risk Regionscited by 6
- 2019Hard but Robust, Easy but Sensitive: How Encoder and Decoder Perform in Neural Machine Translationcited by 5
- 2022Life-long Learning for Multilingual Neural Machine Translation with Knowledge Distillationcited by 5
- 2023Mitigating Data Imbalance and Representation Degeneration in Multilingual Machine Translationcited by 5
- 2023On Synthetic Data for Back Translationcited by 4
- 2022Extending the Subwording Model of Multilingual Pretrained Models for New Languagescited by 3
- 2023Efficiently Upgrading Multilingual Machine Translation Models to Support More Languagescited by 2
- 2022Text Editing as Imitation Gamecited by 1
Built on by
Citing papers per year: 2024: 2; 2025: 1.
3 papers.