# Exploiting Cross-Sentence Context for Neural Machine Translation

arXiv: https://arxiv.org/abs/1704.04347 · HTML: https://mlsanity.com/paper/1704.04347/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Apr 2017.

## Built on by (most cited first, 30 of 48)

- [Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense](https://mlsanity.com/paper/2303.13408/) (2023)
- [Macaw-LLM: Multi-Modal Language Modeling with Image, Audio, Video, and Text Integration](https://mlsanity.com/paper/2306.09093/) (2023)
- [Document-Level Machine Translation with Large Language Models](https://mlsanity.com/paper/2304.02210/) (2023)
- [Uncertainty in Natural Language Generation: From Theory to Applications](https://mlsanity.com/paper/2307.15703/) (2023)
- [Adapting Large Language Models for Document-Level Machine Translation](https://mlsanity.com/paper/2401.06468/) (2024)
- [(Perhaps) Beyond Human Translation: Harnessing Multi-Agent Collaboration for Translating Ultra-Long Literary Texts](https://mlsanity.com/paper/2405.11804/) (2024)
- [Measuring and Increasing Context Usage in Context-Aware Machine Translation](https://mlsanity.com/paper/2105.03482/) (2021)
- [Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Survey](https://mlsanity.com/paper/2104.06951/) (2021)
- [DelTA: An Online Document-Level Translation Agent Based on Multi-Level Memory](https://mlsanity.com/paper/2410.08143/) (2024)
- [Findings of the WMT 2023 Shared Task on Discourse-Level Literary Translation: A Fresh Orb in the Cosmos of LLMs](https://mlsanity.com/paper/2311.03127/) (2023)
- [Do Context-Aware Translation Models Pay the Right Attention?](https://mlsanity.com/paper/2105.06977/) (2021)
- [A Paradigm Shift: The Future of Machine Translation Lies with Large Language Models](https://mlsanity.com/paper/2305.01181/) (2023)
- [Modeling Bilingual Conversational Characteristics for Neural Chat Translation](https://mlsanity.com/paper/2107.11164/) (2021)
- [TranSmart: A Practical Interactive Machine Translation System](https://mlsanity.com/paper/2105.13072/) (2021)
- [Salute the Classic: Revisiting Challenges of Machine Translation in the Age of Large Language Models](https://mlsanity.com/paper/2401.08350/) (2024)
- [Findings of the WMT 2024 Shared Task on Discourse-Level Literary Translation](https://mlsanity.com/paper/2412.11732/) (2024)
- [Diverse Pretrained Context Encodings Improve Document Translation](https://mlsanity.com/paper/2106.03717/) (2021)
- [Hierarchical Transformer-based Large-Context End-to-end ASR with Large-Context Knowledge Distillation](https://mlsanity.com/paper/2102.07935/) (2021)
- [Towards Making the Most of Dialogue Characteristics for Neural Chat Translation](https://mlsanity.com/paper/2109.00668/) (2021)
- [Disco-Bench: A Discourse-Aware Evaluation Benchmark for Language Modelling](https://mlsanity.com/paper/2307.08074/) (2023)
- [A Survey on Zero Pronoun Translation](https://mlsanity.com/paper/2305.10196/) (2023)
- [On Measuring Context Utilization in Document-Level MT Systems](https://mlsanity.com/paper/2402.01404/) (2024)
- [Conversational Speech Recognition By Learning Conversation-level Characteristics](https://mlsanity.com/paper/2202.07855/) (2022)
- [Improving Long Context Document-Level Machine Translation](https://mlsanity.com/paper/2306.05183/) (2023)
- [A Comparison of Approaches to Document-level Machine Translation](https://mlsanity.com/paper/2101.11040/) (2021)
- [DeMPT: Decoding-enhanced Multi-phase Prompt Tuning for Making LLMs Be Better Context-aware Translators](https://mlsanity.com/paper/2402.15200/) (2024)
- [P-Transformer: Towards Better Document-to-Document Neural Machine Translation](https://mlsanity.com/paper/2212.05830/) (2022)
- [Context-aware attention layers coupled with optimal transport domain adaptation and multimodal fusion methods for recognizing dementia from spontaneous speech](https://mlsanity.com/paper/2305.16406/) (2023)
- [A Bilingual Parallel Corpus with Discourse Annotations](https://mlsanity.com/paper/2210.14667/) (2022)
- [HanoiT: Enhancing Context-aware Translation via Selective Context](https://mlsanity.com/paper/2301.06825/) (2023)

## Abstract (arXiv)

In translation, considering the document as a whole can help to resolve ambiguities and inconsistencies. In this paper, we propose a cross-sentence context-aware approach and investigate the influence of historical contextual information on the performance of neural machine translation (NMT). First, this history is summarized in a hierarchical way. We then integrate the historical representation into NMT in two strategies: 1) a warm-start of encoder and decoder states, and 2) an auxiliary context source for updating decoder states. Experimental results on a large Chinese-English translation task show that our approach significantly improves upon a strong attention-based NMT system by up to +2.1 BLEU points.