# Does Neural Machine Translation Benefit from Larger Context?

arXiv: https://arxiv.org/abs/1704.05135 · HTML: https://mlsanity.com/paper/1704.05135/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Apr 2017.

## Built on by (most cited first, 30 of 38)

- [Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense](https://mlsanity.com/paper/2303.13408/) (2023)
- [Large language models effectively leverage document-level context for literary translation, but critical errors persist](https://mlsanity.com/paper/2304.03245/) (2023)
- [Measuring and Increasing Context Usage in Context-Aware Machine Translation](https://mlsanity.com/paper/2105.03482/) (2021)
- [Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey](https://mlsanity.com/paper/2104.06951/) (2021)
- [Exploring Document-Level Literary Machine Translation with Parallel Paragraphs from World Literature](https://mlsanity.com/paper/2210.14250/) (2022)
- [Do Context-Aware Translation Models Pay the Right Attention?](https://mlsanity.com/paper/2105.06977/) (2021)
- [Contextual Refinement of Translations: Large Language Models for Sentence and Document-Level Post-Editing](https://mlsanity.com/paper/2310.14855/) (2023)
- [Modeling Bilingual Conversational Characteristics for Neural Chat Translation](https://mlsanity.com/paper/2107.11164/) (2021)
- [Diverse Pretrained Context Encodings Improve Document Translation](https://mlsanity.com/paper/2106.03717/) (2021)
- [Divide and Rule: Effective Pre-Training for Context-Aware Multi-Encoder Translation Models](https://mlsanity.com/paper/2103.17151/) (2021)
- [Towards Making the Most of Dialogue Characteristics for Neural Chat Translation](https://mlsanity.com/paper/2109.00668/) (2021)
- [Document-Level Language Models for Machine Translation](https://mlsanity.com/paper/2310.12303/) (2023)
- [Data Augmentation by Concatenation for Low-Resource Translation: A Mystery and a Solution](https://mlsanity.com/paper/2105.01691/) (2021)
- [Contrastive Learning for Context-aware Neural Machine TranslationUsing Coreference Information](https://mlsanity.com/paper/2109.05712/) (2021)
- [DOCmT5: Document-Level Pretraining of Multilingual Language Models](https://mlsanity.com/paper/2112.08709/) (2021)
- [Improving Long Context Document-Level Machine Translation](https://mlsanity.com/paper/2306.05183/) (2023)
- [A Comparison of Approaches to Document-level Machine Translation](https://mlsanity.com/paper/2101.11040/) (2021)
- [Multilingual Document-Level Translation Enables Zero-Shot Transfer From Sentences to Documents](https://mlsanity.com/paper/2109.10341/) (2021)
- [DeMPT: Decoding-enhanced Multi-phase Prompt Tuning for Making LLMs Be Better Context-aware Translators](https://mlsanity.com/paper/2402.15200/) (2024)
- [Extending Automatic Machine Translation Evaluation to Book-Length Documents](https://mlsanity.com/paper/2509.17249/) (2025)
- [P-Transformer: Towards Better Document-to-Document Neural Machine Translation](https://mlsanity.com/paper/2212.05830/) (2022)
- [Is Context Helpful for Chat Translation Evaluation?](https://mlsanity.com/paper/2403.08314/) (2024)
- [HanoiT: Enhancing Context-aware Translation via Selective Context](https://mlsanity.com/paper/2301.06825/) (2023)
- [Analyzing the Attention Heads for Pronoun Disambiguation in Context-aware Machine Translation Models](https://mlsanity.com/paper/2412.11187/) (2024)
- [Zero-pronoun Data Augmentation for Japanese-to-English Translation](https://mlsanity.com/paper/2107.00318/) (2021)
- [A Multi-task Multi-stage Transitional Training Framework for Neural Chat Translation](https://mlsanity.com/paper/2301.11749/) (2023)
- [You Are What You Train: Effects of Data Composition on Training Context-aware Machine Translation Models](https://mlsanity.com/paper/2509.14031/) (2025)
- [Sequence Shortening for Context-Aware Machine Translation](https://mlsanity.com/paper/2402.01416/) (2024)
- [Context-Aware Machine Translation with Source Coreference Explanation](https://mlsanity.com/paper/2404.19505/) (2024)
- [GRAFT: A Graph-based Flow-aware Agentic Framework for Document-level Machine Translation](https://mlsanity.com/paper/2507.03311/) (2025)

## Abstract (arXiv)

We propose a neural machine translation architecture that models the surrounding text in addition to the source sentence. These models lead to better performance, both in terms of general translation quality and pronoun prediction, when trained on small corpora, although this improvement largely disappears when trained with a larger corpus. We also discover that attention-based neural machine translation is well suited for pronoun prediction and compares favorably with other approaches that were specifically designed for this task.