Data to 14 Oct 2026

Improving the Transformer Translation Model with Document-Level Context

arXiv 1810.03581 ↗First posted 8 Oct 2018cs.CLcited by 74

Not on the map. Why?

Abstract (arXiv)

Although the Transformer translation model (Vaswani et al., 2017) has achieved state-of-the-art performance in a variety of translation tasks, how to use document-level context to deal with discourse phenomena problematic for Transformer still remains a challenge. In this work, we extend the Transformer model with a new context encoder to represent document-level context, which is then incorporated into the original encoder and decoder. As large-scale document-level parallel corpora are usually not available, we introduce a two-step training method to take full advantage of abundant sentence-level parallel corpora and limited document-level parallel corpora. Experiments on the NIST Chinese-English datasets and the IWSLT French-English datasets show that our approach improves over Transformer significantly.

Key facts. Cited by 74 papers in the map (data through 14 Oct 2026). First posted on arXiv 8 Oct 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 23; 2022: 14; 2023: 19; 2024: 9; 2025: 8; 2026: 1.

74 papers.

Sort:

By area:

2023Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defensecited by 429
2021Do Long-Range Language Models Actually Use Long-Range Context?cited by 92
2023Large language models effectively leverage document-level context for literary translation, but critical errors persistcited by 80
2023Uncertainty in Natural Language Generation: From Theory to Applicationscited by 61
2022Cross-modal Prototype Driven Network for Radiology Report Generationcited by 59
2024Adapting Large Language Models for Document-Level Machine Translationcited by 59
2021BlonDe: An Automatic Evaluation Metric for Document-level Machine Translationcited by 58
2021Measuring and Increasing Context Usage in Context-Aware Machine Translationcited by 48
2021G-Transformer for Document-level Machine Translationcited by 46
2022Exploring Document-Level Literary Machine Translation with Parallel Paragraphs from World Literaturecited by 45
2021Not All Memories are Created Equal: Learning to Forget by Expiringcited by 32
2023Escaping the sentence-level paradigm in machine translationcited by 27

38 more

2024DelTA: An Online Document-Level Translation Agent Based on Multi-Level Memorycited by 27
2021Do Context-Aware Translation Models Pay the Right Attention?cited by 21
2021Centroid Transformers: Learning to Abstract with Attentioncited by 20
2025Benchmarking Vision-Language Models on Chinese Ancient Documents: From OCR to Knowledge Reasoningcited by 19
2021Modeling Bilingual Conversational Characteristics for Neural Chat Translationcited by 18
2022Learn To Remember: Transformer with Recurrent Memory for Document-Level Machine Translationcited by 16
2023Document Flattening: Beyond Concatenating Context for Document-Level Neural Machine Translationcited by 15
2021Diverse Pretrained Context Encodings Improve Document Translationcited by 13
2024Enhancing Document-level Translation of Large Language Model via Translation Mixed-instructionscited by 13
2021Hierarchical Transformer-based Large-Context End-to-end ASR with Large-Context Knowledge Distillationcited by 12
2021Divide and Rule: Effective Pre-Training for Context-Aware Multi-Encoder Translation Modelscited by 12
2021Towards Making the Most of Dialogue Characteristics for Neural Chat Translationcited by 12
2023Document-Level Language Models for Machine Translationcited by 12
2021Data Augmentation by Concatenation for Low-Resource Translation: A Mystery and a Solutioncited by 11
2025Multilingual Contextualization of Large Language Models for Document-Level Machine Translationcited by 11
2024Investigating Length Issues in Document-level Machine Translationcited by 10
2021Contrastive Learning for Context-aware Neural Machine TranslationUsing Coreference Informationcited by 9
2022Integrating Vectorized Lexical Constraints for Neural Machine Translationcited by 9
2021DOCmT5: Document-Level Pretraining of Multilingual Language Modelscited by 8
2023Improving Long Context Document-Level Machine Translationcited by 8
2021A Comparison of Approaches to Document-level Machine Translationcited by 7
2021Multilingual Document-Level Translation Enables Zero-Shot Transfer From Sentences to Documentscited by 7
2022Scheduled Multi-task Learning for Neural Chat Translationcited by 7
2022Focused Concatenation for Context-Aware Neural Machine Translationcited by 7
2023Discourse Centric Evaluation of Machine Translation with a Densely Annotated Parallel Corpuscited by 7
2025DocHPLT: A Massively Multilingual Document-Level Translation Datasetcited by 7
2021Multi-Encoder Learning and Stream Fusion for Transformer-Based End-to-End Automatic Speech Recognitioncited by 6
2024DeMPT: Decoding-enhanced Multi-phase Prompt Tuning for Making LLMs Be Better Context-aware Translatorscited by 6
2022CUE Vectors: Modular Training of Language Models Conditioned on Diverse Contextual Signalscited by 5
2022Dodging the Data Bottleneck: Automatic Subtitling with Automatically Segmented ST Corporacited by 5
2022The Effect of Normalization for Bi-directional Amharic-English Neural Machine Translationcited by 5
2022P-Transformer: Towards Better Document-to-Document Neural Machine Translationcited by 5
2023Challenges in Context-Aware Neural Machine Translationcited by 5
2021Towards Personalised and Document-level Machine Translation of Dialoguecited by 4
2022A Bilingual Parallel Corpus with Discourse Annotationscited by 4
2023HanoiT: Enhancing Context-aware Translation via Selective Contextcited by 4
2024A Case Study on Context-Aware Neural Machine Translation with Multi-Task Learningcited by 4
2022Modeling Context With Linear Attention for Scalable Document-Level Translationcited by 3

50 most cited of 74 All 74 (JSON)