# Vocabulary Manipulation for Neural Machine Translation

arXiv: https://arxiv.org/abs/1605.03209 · HTML: https://mlsanity.com/paper/1605.03209/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 May 2016.

## Built on by (most cited first, 5 of 5)

- [Tokenization and the Noiseless Channel](https://mlsanity.com/paper/2306.16842/) (2023)
- [Dynamic Language Models for Continuously Evolving Content](https://mlsanity.com/paper/2106.06297/) (2021)
- [On-the-Fly Fusion of Large Language Models and Machine Translation](https://mlsanity.com/paper/2311.08306/) (2023)
- [The Devil is in the Details: On the Pitfalls of Vocabulary Selection in Neural Machine Translation](https://mlsanity.com/paper/2205.06618/) (2022)
- [Word Alignment in the Era of Deep Learning: A Tutorial](https://mlsanity.com/paper/2212.00138/) (2022)

## Abstract (arXiv)

In order to capture rich language phenomena, neural machine translation models have to use a large vocabulary size, which requires high computing time and large memory usage. In this paper, we alleviate this issue by introducing a sentence-level or batch-level vocabulary, which is only a very small sub-set of the full output vocabulary. For each sentence or batch, we only predict the target words in its sentence-level or batch-level vocabulary. Thus, we reduce both the computing time and the memory usage. Our method simply takes into account the translation options of each word or phrase in the source sentence, and picks a very small target vocabulary for each sentence based on a word-to-word translation model or a bilingual phrase library learned from a traditional machine translation model. Experimental results on the large-scale English-to-French task show that our method achieves better translation performance by 1 BLEU point over the large vocabulary neural machine translation system of Jean et al. (2015).