# Improving Word Sense Disambiguation in Neural Machine Translation with Salient Document Context

arXiv: https://arxiv.org/abs/2311.15507 · HTML: https://mlsanity.com/paper/2311.15507/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 51 references that are in the map.
- First posted on arXiv 27 Nov 2023.

## Builds on (most cited first, 21 of 21)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [COMET: A Neural Framework for MT Evaluation](https://mlsanity.com/paper/2009.09025/) (2020)
- [Subword Regularization: Improving Neural Network Translation Models with Multiple Subword Candidates](https://mlsanity.com/paper/1804.10959/) (2018)
- [Language-agnostic BERT Sentence Embedding](https://mlsanity.com/paper/2007.01852/) (2020)
- [Neural Machine Translation with Extended Context](https://mlsanity.com/paper/1708.05943/) (2017)
- [Evaluating Discourse Phenomena in Neural Machine Translation](https://mlsanity.com/paper/1711.00513/) (2017)
- [Document-Level Neural Machine Translation with Hierarchical Attention Networks](https://mlsanity.com/paper/1809.01576/) (2018)
- [BlonDe: An Automatic Evaluation Metric for Document-level Machine Translation](https://mlsanity.com/paper/2103.11878/) (2021)
- [Measuring and Increasing Context Usage in Context-Aware Machine Translation](https://mlsanity.com/paper/2105.03482/) (2021)
- [Learning to Remember Translation History with a Continuous Cache](https://mlsanity.com/paper/1711.09367/) (2017)
- [Does Multi-Encoder Help? A Case Study on Context-Aware Neural Machine Translation](https://mlsanity.com/paper/2005.03393/) (2020)
- [Handling Homographs in Neural Machine Translation](https://mlsanity.com/paper/1708.06510/) (2017)
- [Embedding Words and Senses Together via Joint Knowledge-Enhanced Training](https://mlsanity.com/paper/1612.02703/) (2016)
- [Using Whole Document Context in Neural Machine Translation](https://mlsanity.com/paper/1910.07481/) (2019)
- [Integrating Weakly Supervised Word Sense Disambiguation into Neural Machine Translation](https://mlsanity.com/paper/1810.02614/) (2018)
- [Context in Neural Machine Translation: A Review of Models and Evaluations](https://mlsanity.com/paper/1901.09115/) (2019)
- [Fusing Recency into Neural Machine Translation with an Inter-Sentence Gate Model](https://mlsanity.com/paper/1806.04466/) (2018)
- [A Test Suite and Manual Evaluation of Document-Level NMT at WMT19](https://mlsanity.com/paper/1908.03043/) (2019)

## Built on by (most cited first, 2 of 2)

- [QueryBandits for Hallucination Mitigation: Exploiting Semantic Features for No-Regret Rewriting](https://mlsanity.com/paper/2508.16697/) (2025)
- [No One Size Fits All: QueryBandits for Hallucination Mitigation](https://mlsanity.com/paper/2602.20332/) (2026)

## Abstract (arXiv)

Lexical ambiguity is a challenging and pervasive problem in machine translation (mt). We introduce a simple and scalable approach to resolve translation ambiguity by incorporating a small amount of extra-sentential context in neural mt. Our approach requires no sense annotation and no change to standard model architectures. Since actual document context is not available for the vast majority of mt training data, we collect related sentences for each input to construct pseudo-documents. Salient words from pseudo-documents are then encoded as a prefix to each source sentence to condition the generation of the translation. To evaluate, we release docmucow, a challenge set for translation disambiguation based on the English-German mucow citeraganato-etal-2020-evaluation augmented with document IDs. Extensive experiments show that our method translates ambiguous source words better than strong sentence-level baselines and comparable document-level baselines while reducing training costs.