Improving Word Sense Disambiguation in Neural Machine Translation with Salient Document Context

Lexical ambiguity is a challenging and pervasive problem in machine translation (mt). We introduce a simple and scalable approach to resolve translation ambiguity by incorporating a small amount of extra-sentential context in neural mt. Our approach requires no sense annotation and no change to standard model architectures. Since actual document context is not available for the vast majority of mt training data, we collect related sentences for each input to construct pseudo-documents. Salient words from pseudo-documents are then encoded as a prefix to each source sentence to condition the generation of the translation. To evaluate, we release docmucow, a challenge set for translation disambiguation based on the English-German mucow citeraganato-etal-2020-evaluation augmented with document IDs. Extensive experiments show that our method translates ambiguous source words better than strong sentence-level baselines and comparable document-level baselines while reducing training costs.