# Continuous Learning in Neural Machine Translation using Bilingual Dictionaries

arXiv: https://arxiv.org/abs/2102.06558 · HTML: https://mlsanity.com/paper/2102.06558/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 30 references that are in the map.
- First posted on arXiv 12 Feb 2021.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Neural Machine Translation of Rare Words with Subword Units](https://mlsanity.com/paper/1508.07909/) (2015)
- [Stanza: A Python Natural Language Processing Toolkit for Many Human Languages](https://mlsanity.com/paper/2003.07082/) (2020)
- [Lexically Constrained Decoding for Sequence Generation Using Grid Beam Search](https://mlsanity.com/paper/1704.07138/) (2017)
- [Fast Lexically Constrained Decoding with Dynamic Beam Allocation for Neural Machine Translation](https://mlsanity.com/paper/1804.06609/) (2018)
- [A Theoretically Grounded Application of Dropout in Recurrent Neural Networks](https://mlsanity.com/paper/1512.05287/) (2015)
- [Training Neural Machine Translation To Apply Terminology Constraints](https://mlsanity.com/paper/1906.01105/) (2019)
- [Very Deep Self-Attention Networks for End-to-End Speech Recognition](https://mlsanity.com/paper/1904.13377/) (2019)
- [How Grammatical is Character-level Neural Machine Translation? Assessing MT Quality with Contrastive Translation Pairs](https://mlsanity.com/paper/1612.04629/) (2016)
- [Domain Control for Neural Machine Translation](https://mlsanity.com/paper/1612.06140/) (2016)
- [Neural Machine Translation Decoding with Terminology Constraints](https://mlsanity.com/paper/1805.03750/) (2018)
- [Incorporating Discrete Translation Lexicons into Neural Machine Translation](https://mlsanity.com/paper/1606.02006/) (2016)
- [Bridging Neural Machine Translation and Bilingual Dictionaries](https://mlsanity.com/paper/1610.07272/) (2016)
- [Towards one-shot learning for rare-word translation with external experts](https://mlsanity.com/paper/1809.03182/) (2018)
- [Pre-Translation for Neural Machine Translation](https://mlsanity.com/paper/1610.05243/) (2016)
- [Prior Knowledge Integration for Neural Machine Translation using Posterior Regularization](https://mlsanity.com/paper/1811.01100/) (2018)
- [Learning Efficient Lexically-Constrained Neural Machine Translation with External Memory](https://mlsanity.com/paper/1901.11344/) (2019)

## Built on by (most cited first, 11 of 11)

- [Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey](https://mlsanity.com/paper/2104.06951/) (2021)
- [Bilex Rx: Lexical Data Augmentation for Massively Multilingual Machine Translation](https://mlsanity.com/paper/2303.15265/) (2023)
- [Dynamic Terminology Integration for COVID-19 and other Emerging Domains](https://mlsanity.com/paper/2109.04708/) (2021)
- [Towards Accurate Translation via Semantically Appropriate Application of Lexical Constraints](https://mlsanity.com/paper/2306.12089/) (2023)
- [Optimizing Rare Word Accuracy in Direct Speech Translation with a Retrieval-and-Demonstration Approach](https://mlsanity.com/paper/2409.09009/) (2024)
- [Cost-Effective Training in Low-Resource Neural Machine Translation](https://mlsanity.com/paper/2201.05700/) (2022)
- [Improving Rare Word Translation With Dictionaries and Attention Masking](https://mlsanity.com/paper/2408.09075/) (2024)
- [Modeling Target-side Inflection in Placeholder Translation](https://mlsanity.com/paper/2107.00334/) (2021)
- [Towards Global AI Inclusivity: A Large-Scale Multilingual Terminology Dataset (GIST)](https://mlsanity.com/paper/2412.18367/) (2024)
- [A Template-based Method for Constrained Neural Machine Translation](https://mlsanity.com/paper/2205.11255/) (2022)
- [TULUN: Transparent and Adaptable Low-resource Machine Translation](https://mlsanity.com/paper/2505.18683/) (2025)

## Abstract (arXiv)

While recent advances in deep learning led to significant improvements in machine translation, neural machine translation is often still not able to continuously adapt to the environment. For humans, as well as for machine translation, bilingual dictionaries are a promising knowledge source to continuously integrate new knowledge. However, their exploitation poses several challenges: The system needs to be able to perform one-shot learning as well as model the morphology of source and target language. In this work, we proposed an evaluation framework to assess the ability of neural machine translation to continuously learn new phrases. We integrate one-shot learning methods for neural machine translation with different word representations and show that it is important to address both in order to successfully make use of bilingual dictionaries. By addressing both challenges we are able to improve the ability to translate new, rare words and phrases from 30% to up to 70%. The correct lemma is even generated by more than 90%.