mlsanityData to 7 Oct 2026

Towards Continual Learning for Multilingual Machine Translation via Vocabulary Substitution

arXiv 2103.06799 ↗First posted 11 Mar 2021cs.CLcited by 31

Not on the map. Why?

Abstract (arXiv)

We propose a straightforward vocabulary adaptation scheme to extend the language capacity of multilingual machine translation models, paving the way towards efficient continual learning for multilingual machine translation. Our approach is suitable for large-scale datasets, applies to distant languages with unseen scripts, incurs only minor degradation on the translation performance for the original language pairs and provides competitive performance even in the case where we only possess monolingual data for the new languages.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). Builds on 27 of its 47 references that are in the map. First posted on arXiv 11 Mar 2021.

Builds on

27 of 47 references.

References · 27

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2016Overcoming catastrophic forgetting in neural networkscited by 4,718
  4. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  5. 2016iCaRL: Incremental Classifier and Representation Learningcited by 2,313
  6. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  7. 2017Gradient Episodic Memory for Continual Learningcited by 1,871
  8. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  9. 2020Multilingual Denoising Pre-training for Neural Machine Translationcited by 1,028
  10. 2015Improving Neural Machine Translation Models with Monolingual Datacited by 784
  11. 2016Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translationcited by 467
  12. 2019MASS: Masked Sequence to Sequence Pre-training for Language Generationcited by 345
15 more
  1. 2020Multilingual Translation with Extensible Multilingual Pretraining and Finetuningcited by 287
  2. 2019Massively Multilingual Neural Machine Translation in the Wild: Findings and Challengescited by 253
  3. 2019Massively Multilingual Neural Machine Translationcited by 246
  4. 2016Transfer Learning for Low-Resource Neural Machine Translationcited by 188
  5. 2018When and Why are Pre-trained Word Embeddings Useful for Neural Machine Translation?cited by 125
  6. 2016Multi-Way, Multilingual Neural Machine Translation with a Shared Attention Mechanismcited by 117
  7. 2018Rapid Adaptation of Neural Machine Translation to New Languagescited by 82
  8. 2018Universal Neural Machine Translation for Extremely Low Resource Languagescited by 66
  9. 2020Leveraging Monolingual Data with Self-Supervision for Multilingual Neural Machine Translationcited by 51
  10. 2020A Multilingual View of Unsupervised Machine Translationcited by 23
  11. 2018Transfer Learning in Multilingual Neural Machine Translation with Dynamic Vocabularycited by 18
  12. 2020Reusing a Pretrained Language Model on Languages with Limited Corpora for Unsupervised NMTcited by 18
  13. 2019From Bilingual to Multilingual Neural Machine Translation by Incremental Trainingcited by 14
  14. 2019Adapting Multilingual Neural Machine Translation to Unseen Languagescited by 12
  15. 2019Towards Interlingua Neural Machine Translationcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 4; 2022: 12; 2023: 8; 2024: 5; 2025: 1; 2026: 1.

31 papers.

Sort:

By area:

  1. 2023A Comprehensive Survey of Continual Learning: Theory, Method and Applicationcited by 860
  2. 2021ByT5: Towards a token-free future with pre-trained byte-to-byte modelscited by 504
  3. 2022Lifting the Curse of Multilinguality by Pre-training Modular Transformerscited by 145
  4. 2022Building Machine Translation Systems for the Next Thousand Languagescited by 69
  5. 2024Towards Lifelong Learning of Large Language Models: A Surveycited by 60
  6. 2023mmT5: Modular Multilingual Pre-Training Solves Source Language Hallucinationscited by 24
  7. 2021Multilingual Domain Adaptation for NMT: Decoupling Language and Domain Information with Adapterscited by 21
  8. 2024An Empirical Comparison of Vocabulary Expansion and Initialization Approaches for Language Modelscited by 20
  9. 2022Controlling Translation Formality Using Pre-trained Multilingual Language Modelscited by 14
  10. 2021Continual Learning in Multilingual NMT via Language-Specific Embeddingscited by 12
  11. 2023Tik-to-Tok: Translating Language Models One Token at a Time: An Embedding Initialization Strategy for Efficient Language Adaptationcited by 11
  12. 2022Cross-lingual Lifelong Learningcited by 9
19 more
  1. 2022Parameter-Efficient Finetuning for Robust Continual Multilingual Learningcited by 9
  2. 2023Continual Learning Under Language Shiftcited by 9
  3. 2022Tencent AI Lab - Shanghai Jiao Tong University Low-Resource Translation System for the WMT22 Translation Taskcited by 8
  4. 2023Multilingual Sentence Transformer as A Multilingual Word Alignercited by 8
  5. 2024Embracing Language Inclusivity and Diversity in CLIP through Continual Language Learningcited by 7
  6. 2022Training a T5 Using Lab-sized Resourcescited by 5
  7. 2022Dodging the Data Bottleneck: Automatic Subtitling with Automatically Segmented ST Corporacited by 5
  8. 2022Continual Learning of Long Topic Sequences in Neural Information Retrievalcited by 4
  9. 2022Continual Knowledge Distillation for Neural Machine Translationcited by 4
  10. 2023Extending Multilingual Machine Translation through Imitation Learningcited by 3
  11. 2024BinaryAlign: Word Alignment as Binary Sequence Labelingcited by 3
  12. 2022MANTa: Efficient Gradient-Based Tokenization for Robust End-to-End Language Modelingcited by 2
  13. 2023Efficiently Upgrading Multilingual Machine Translation Models to Support More Languagescited by 2
  14. 2023How Transferable are Attribute Controllers on Pretrained Multilingual Translation Models?cited by 2
  15. 2025BanglaByT5: Byte-Level Modelling for Banglacited by 2
  16. 2021Multilingual unsupervised sequence segmentation transfers to extremely low-resource languagescited by 1
  17. 2022On Robust Incremental Learning over Many Multilingual Stepscited by 0
  18. 2024Decoupled Vocabulary Learning Enables Zero-Shot Translation from Unseen Languagescited by 0
  19. 2026Towards Reliable Multilingual LLMs-as-a-Judge: An Empirical Studycited by 0