# Neural Machine Translation with Extended Context

arXiv: https://arxiv.org/abs/1708.05943 · HTML: https://mlsanity.com/paper/1708.05943/

## Key facts

- Cited by 95 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Aug 2017.

## Built on by (most cited first, 30 of 95)

- [Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense](https://mlsanity.com/paper/2303.13408/) (2023)
- [Large language models effectively leverage document-level context for literary translation, but critical errors persist](https://mlsanity.com/paper/2304.03245/) (2023)
- [Adapting Large Language Models for Document-Level Machine Translation](https://mlsanity.com/paper/2401.06468/) (2024)
- [On the Blind Spots of Model-Based Evaluation Metrics for Text Generation](https://mlsanity.com/paper/2212.10020/) (2022)
- [Measuring and Increasing Context Usage in Context-Aware Machine Translation](https://mlsanity.com/paper/2105.03482/) (2021)
- [Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey](https://mlsanity.com/paper/2104.06951/) (2021)
- [G-Transformer for Document-level Machine Translation](https://mlsanity.com/paper/2105.14761/) (2021)
- [Exploring Document-Level Literary Machine Translation with Parallel Paragraphs from World Literature](https://mlsanity.com/paper/2210.14250/) (2022)
- [MT-GenEval: A Counterfactual and Contextual Dataset for Evaluating Gender Accuracy in Machine Translation](https://mlsanity.com/paper/2211.01355/) (2022)
- [When Does Translation Require Context? A Data-driven, Multilingual Exploration](https://mlsanity.com/paper/2109.07446/) (2021)
- [Escaping the sentence-level paradigm in machine translation](https://mlsanity.com/paper/2304.12959/) (2023)
- [Do Context-Aware Translation Models Pay the Right Attention?](https://mlsanity.com/paper/2105.06977/) (2021)
- [Contextual Refinement of Translations: Large Language Models for Sentence and Document-Level Post-Editing](https://mlsanity.com/paper/2310.14855/) (2023)
- [Investigating Failures of Automatic Translation in the Case of Unambiguous Gender](https://mlsanity.com/paper/2104.07838/) (2021)
- [Modeling Bilingual Conversational Characteristics for Neural Chat Translation](https://mlsanity.com/paper/2107.11164/) (2021)
- [How "Real" is Your Real-Time Simultaneous Speech-to-Text Translation System?](https://mlsanity.com/paper/2412.18495/) (2024)
- [Learn To Remember: Transformer with Recurrent Memory for Document-Level Machine Translation](https://mlsanity.com/paper/2205.01546/) (2022)
- [Document Flattening: Beyond Concatenating Context for Document-Level Neural Machine Translation](https://mlsanity.com/paper/2302.08079/) (2023)
- [Diverse Pretrained Context Encodings Improve Document Translation](https://mlsanity.com/paper/2106.03717/) (2021)
- [Enhancing Document-level Translation of Large Language Model via Translation Mixed-instructions](https://mlsanity.com/paper/2401.08088/) (2024)
- [Understanding In-Context Machine Translation for Low-Resource Languages: A Case Study on Manchu](https://mlsanity.com/paper/2502.11862/) (2025)
- [Divide and Rule: Effective Pre-Training for Context-Aware Multi-Encoder Translation Models](https://mlsanity.com/paper/2103.17151/) (2021)
- [Towards Making the Most of Dialogue Characteristics for Neural Chat Translation](https://mlsanity.com/paper/2109.00668/) (2021)
- [Document-Level Language Models for Machine Translation](https://mlsanity.com/paper/2310.12303/) (2023)
- [Data Augmentation by Concatenation for Low-Resource Translation: A Mystery and a Solution](https://mlsanity.com/paper/2105.01691/) (2021)
- [A baseline revisited: Pushing the limits of multi-segment models for context-aware translation](https://mlsanity.com/paper/2210.10906/) (2022)
- [AFRIDOC-MT: Document-level MT Corpus for African Languages](https://mlsanity.com/paper/2501.06374/) (2025)
- [Towards User-Driven Neural Machine Translation](https://mlsanity.com/paper/2106.06200/) (2021)
- [Investigating Length Issues in Document-level Machine Translation](https://mlsanity.com/paper/2412.17592/) (2024)
- [Contrastive Learning for Context-aware Neural Machine TranslationUsing Coreference Information](https://mlsanity.com/paper/2109.05712/) (2021)

## Abstract (arXiv)

We investigate the use of extended context in attention-based neural machine translation. We base our experiments on translated movie subtitles and discuss the effect of increasing the segments beyond single translation units. We study the use of extended source language context as well as bilingual context extensions. The models learn to distinguish between information from different segments and are surprisingly robust with respect to translation quality. In this pilot study, we observe interesting cross-sentential attention patterns that improve textual coherence in translation at least in some selected cases.