# Rapid Adaptation of Neural Machine Translation to New Languages

arXiv: https://arxiv.org/abs/1808.04189 · HTML: https://mlsanity.com/paper/1808.04189/

## Key facts

- Cited by 82 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Aug 2018.

## Built on by (most cited first, 30 of 82)

- [Neural Machine Translation for Low-Resource Languages: A Survey](https://mlsanity.com/paper/2106.15115/) (2021)
- [Survey of Low-Resource Machine Translation](https://mlsanity.com/paper/2109.00486/) (2021)
- [BLOOM+1: Adding Language Support to BLOOM for Zero-Shot Prompting](https://mlsanity.com/paper/2212.09535/) (2022)
- [Cross-Attention is All You Need: Adapting Pretrained Transformers for Machine Translation](https://mlsanity.com/paper/2104.08771/) (2021)
- [Building Machine Translation Systems for the Next Thousand Languages](https://mlsanity.com/paper/2205.03983/) (2022)
- [Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey](https://mlsanity.com/paper/2104.06951/) (2021)
- [SEACrowd: A Multilingual Multimodal Data Hub and Benchmark Suite for Southeast Asian Languages](https://mlsanity.com/paper/2406.10118/) (2024)
- [Multilingual Multimodal Pre-training for Zero-Shot Cross-Lingual Transfer of Vision-Language Models](https://mlsanity.com/paper/2103.08849/) (2021)
- [Towards Continual Learning for Multilingual Machine Translation via Vocabulary Substitution](https://mlsanity.com/paper/2103.06799/) (2021)
- [Many-to-English Machine Translation Tools, Data, and Pretrained Models](https://mlsanity.com/paper/2104.00290/) (2021)
- [Making More of Little Data: Improving Low-Resource Automatic Speech Recognition Using Data Augmentation](https://mlsanity.com/paper/2305.10951/) (2023)
- [AfroMT: Pretraining Strategies and Reproducible Benchmarks for Translation of 8 African Languages](https://mlsanity.com/paper/2109.04715/) (2021)
- [Pre-Trained Multilingual Sequence-to-Sequence Models: A Hope for Low-Resource Language Translation?](https://mlsanity.com/paper/2203.08850/) (2022)
- [Multilingual Domain Adaptation for NMT: Decoupling Language and Domain Information with Adapters](https://mlsanity.com/paper/2110.09574/) (2021)
- [Lego-MT: Learning Detachable Models for Massively Multilingual Machine Translation](https://mlsanity.com/paper/2212.10551/) (2022)
- [Exploiting Language Relatedness for Low Web-Resource Language Model Adaptation: An Indic Languages Study](https://mlsanity.com/paper/2106.03958/) (2021)
- [Towards the Next 1000 Languages in Multilingual Machine Translation: Exploring the Synergy Between Supervised and Self-Supervised Learning](https://mlsanity.com/paper/2201.03110/) (2022)
- [Distributionally Robust Multilingual Machine Translation](https://mlsanity.com/paper/2109.04020/) (2021)
- [Adapting High-resource NMT Models to Translate Low-resource Related Languages without Parallel Data](https://mlsanity.com/paper/2105.15071/) (2021)
- [Meta Back-translation](https://mlsanity.com/paper/2102.07847/) (2021)
- [A Survey on Low-Resource Neural Machine Translation](https://mlsanity.com/paper/2107.04239/) (2021)
- [Bandits Don't Follow Rules: Balancing Multi-Facet Machine Translation with Multi-Armed Bandits](https://mlsanity.com/paper/2110.06997/) (2021)
- [SMaLL-100: Introducing Shallow Multilingual Machine Translation Model for Low-Resource Languages](https://mlsanity.com/paper/2210.11621/) (2022)
- [DEEP: DEnoising Entity Pre-training for Neural Machine Translation](https://mlsanity.com/paper/2111.07393/) (2021)
- [Viewing Knowledge Transfer in Multilingual Machine Translation Through a Representational Lens](https://mlsanity.com/paper/2305.11550/) (2023)
- [Language-Family Adapters for Low-Resource Multilingual Neural Machine Translation](https://mlsanity.com/paper/2209.15236/) (2022)
- [Scalable Fine-tuning from Multiple Data Sources: A First-Order Approximation Approach](https://mlsanity.com/paper/2409.19458/) (2024)
- [Continual Learning in Multilingual NMT via Language-Specific Embeddings](https://mlsanity.com/paper/2110.10478/) (2021)
- [How Vocabulary Sharing Facilitates Multilingualism in LLaMA?](https://mlsanity.com/paper/2311.09071/) (2023)
- [MMTAfrica: Multilingual Machine Translation for African Languages](https://mlsanity.com/paper/2204.04306/) (2022)

## Abstract (arXiv)

This paper examines the problem of adapting neural machine translation systems to new, low-resourced languages (LRLs) as effectively and rapidly as possible. We propose methods based on starting with massively multilingual "seed models", which can be trained ahead-of-time, and then continuing training on data related to the LRL. We contrast a number of strategies, leading to a novel, simple, yet effective method of "similar-language regularization", where we jointly train on both a LRL of interest and a similar high-resourced language to prevent over-fitting to small LRL data. Experiments demonstrate that massively multilingual models, even without any explicit adaptation, are surprisingly effective, achieving BLEU scores of up to 15.5 with no data from the LRL, and that the proposed similar-language regularization method improves over other adaptation methods by 1.7 BLEU points average over 4 LRL settings. Code to reproduce experiments at https://github.com/neubig/rapid-adaptation