# Revisiting Context Choices for Context-aware Machine Translation

arXiv: https://arxiv.org/abs/2109.02995 · HTML: https://mlsanity.com/paper/2109.02995/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 37 references that are in the map.
- First posted on arXiv 7 Sep 2021.

## Builds on (most cited first, 22 of 22)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://mlsanity.com/paper/1808.06226/) (2018)
- [A Call for Clarity in Reporting BLEU Scores](https://mlsanity.com/paper/1804.08771/) (2018)
- [Marian: Fast Neural Machine Translation in C++](https://mlsanity.com/paper/1804.00344/) (2018)
- [Context-Aware Neural Machine Translation Learns Anaphora Resolution](https://mlsanity.com/paper/1805.10163/) (2018)
- [Neural Machine Translation with Extended Context](https://mlsanity.com/paper/1708.05943/) (2017)
- [Document-Level Neural Machine Translation with Hierarchical Attention Networks](https://mlsanity.com/paper/1809.01576/) (2018)
- [Improving the Transformer Translation Model with Document-Level Context](https://mlsanity.com/paper/1810.03581/) (2018)
- [Selective Attention for Context-aware Neural Machine Translation](https://mlsanity.com/paper/1903.08788/) (2019)
- [Microsoft Translator at WMT 2019: Towards Large-Scale Document-Level Neural Machine Translation](https://mlsanity.com/paper/1907.06170/) (2019)
- [When and Why is Document-level Context Useful in Neural Machine Translation?](https://mlsanity.com/paper/1910.00294/) (2019)
- [Learning to Remember Translation History with a Continuous Cache](https://mlsanity.com/paper/1711.09367/) (2017)
- [Does Neural Machine Translation Benefit from Larger Context?](https://mlsanity.com/paper/1704.05135/) (2017)
- [Multi-Source Neural Translation](https://mlsanity.com/paper/1601.00710/) (2016)
- [Does Multi-Encoder Help? A Case Study on Context-Aware Neural Machine Translation](https://mlsanity.com/paper/2005.03393/) (2020)
- [Multi-Encoder-Decoder Transformer for Code-Switching Speech Recognition](https://mlsanity.com/paper/2006.10414/) (2020)
- [JParaCrawl: A Large Scale Web-Based English-Japanese Parallel Corpus](https://mlsanity.com/paper/1911.10668/) (2019)
- [MS-UEdin Submission to the WMT2018 APE Shared Task: Dual-Source Transformer for Automatic Post-Editing](https://mlsanity.com/paper/1809.00188/) (2018)
- [Using Whole Document Context in Neural Machine Translation](https://mlsanity.com/paper/1910.07481/) (2019)
- [Document-aligned Japanese-English Conversation Parallel Corpus](https://mlsanity.com/paper/2012.06143/) (2020)
- [Can Your Context-Aware MT System Pass the DiP Benchmark Tests? : Evaluation Benchmarks for Discourse Phenomena in Machine Translation](https://mlsanity.com/paper/2004.14607/) (2020)
- [Addressing Zero-Resource Domains Using Document-Level Context in Neural Machine Translation](https://mlsanity.com/paper/2004.14927/) (2020)

## Built on by (most cited first, 1 of 1)

- [On Measuring Context Utilization in Document-Level MT Systems](https://mlsanity.com/paper/2402.01404/) (2024)

## Abstract (arXiv)

One of the most popular methods for context-aware machine translation (MT) is to use separate encoders for the source sentence and context as multiple sources for one target sentence. Recent work has cast doubt on whether these models actually learn useful signals from the context or are improvements in automatic evaluation metrics just a side-effect. We show that multi-source transformer models improve MT over standard transformer-base models even with empty lines provided as context, but the translation quality improves significantly (1.51 - 2.65 BLEU) when a sufficient amount of correct context is provided. We also show that even though randomly shuffling in-domain context can also improve over baselines, the correct context further improves translation quality and random out-of-domain context further degrades it.