mlsanityData to 7 Oct 2026

Challenges in Context-Aware Neural Machine Translation

arXiv 2305.13751 ↗First posted 23 May 2023cs.CLcited by 5

Not on the map. Why?

Abstract (arXiv)

Context-aware neural machine translation involves leveraging information beyond sentence-level context to resolve inter-sentential discourse dependencies and improve document-level translation quality, and has given rise to a number of recent techniques. However, despite well-reasoned intuitions, most context-aware translation models show only modest improvements over sentence-level systems. In this work, we investigate several challenges that impede progress within this field, relating to discourse phenomena, context usage, model architectures, and document-level evaluation. To address these problems, we propose a more realistic setting for document-level translation, called paragraph-to-paragraph (para2para) translation, and collect a new dataset of Chinese-English novels to promote future research.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 41 of its 75 references that are in the map. First posted on arXiv 23 May 2023.

Builds on

41 of 75 references.

References · 41

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2014Sequence to Sequence Learning with Neural Networkscited by 2,974
  4. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,898
  5. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,973
  6. 2018A Call for Clarity in Reporting BLEU Scorescited by 1,790
  7. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  8. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,471
  9. 2020COMET: A Neural Framework for MT Evaluationcited by 1,024
  10. 2020Efficient Transformers: A Surveycited by 784
  11. 2019Pay Less Attention with Lightweight and Dynamic Convolutionscited by 211
  12. 2022Mega: Moving Average Equipped Gated Attentioncited by 178
29 more
  1. 2018Achieving Human Parity on Automatic Chinese to English News Translationcited by 119
  2. 2018Context-Aware Neural Machine Translation Learns Anaphora Resolutioncited by 97
  3. 2017Neural Machine Translation with Extended Contextcited by 95
  4. 2018Has Machine Translation Achieved Human Parity? A Case for Document-level Evaluationcited by 88
  5. 2017Evaluating Discourse Phenomena in Neural Machine Translationcited by 87
  6. 2018Document-Level Neural Machine Translation with Hierarchical Attention Networkscited by 85
  7. 2023Large language models effectively leverage document-level context for literary translation, but critical errors persistcited by 80
  8. 2018Improving the Transformer Translation Model with Document-Level Contextcited by 74
  9. 2019Selective Attention for Context-aware Neural Machine Translationcited by 60
  10. 2021BlonDe: An Automatic Evaluation Metric for Document-level Machine Translationcited by 58
  11. 2022Embarrassingly Easy Document-Level MT Metrics: How to Convert Any Pretrained Metric Into a Document-Level Metriccited by 56
  12. 2020A Survey of Deep Learning Techniques for Neural Machine Translationcited by 50
  13. 2021Measuring and Increasing Context Usage in Context-Aware Machine Translationcited by 48
  14. 2019When and Why is Document-level Context Useful in Neural Machine Translation?cited by 45
  15. 2022Exploring Document-Level Literary Machine Translation with Parallel Paragraphs from World Literaturecited by 45
  16. 2017Document Context Neural Machine Translation with Memory Networkscited by 41
  17. 2017Learning to Remember Translation History with a Continuous Cachecited by 39
  18. 2020Does Multi-Encoder Help? A Case Study on Context-Aware Neural Machine Translationcited by 34
  19. 2020Rethinking Document-level Neural Machine Translationcited by 32
  20. 2023Escaping the sentence-level paradigm in machine translationcited by 27
  21. 2023Document Flattening: Beyond Concatenating Context for Document-Level Neural Machine Translationcited by 15
  22. 2019Multilingual Graphemic Hybrid ASR with Massive Data Augmentationcited by 13
  23. 2021Divide and Rule: Effective Pre-Training for Context-Aware Multi-Encoder Translation Modelscited by 12
  24. 2021Contrastive Learning for Context-aware Neural Machine TranslationUsing Coreference Informationcited by 9
  25. 2021A Comparison of Approaches to Document-level Machine Translationcited by 7
  26. 2022Focused Concatenation for Context-Aware Neural Machine Translationcited by 7
  27. 2018Fusing Recency into Neural Machine Translation with an Inter-Sentence Gate Modelcited by 5
  28. 2022Modeling Context With Linear Attention for Scalable Document-Level Translationcited by 3
  29. 2022A Bilingual Parallel Corpus with Discourse Annotationscited by 3

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 2; 2025: 1.

5 papers.

  1. 2025Beyond Decoder-only: Large Language Models Can be Good Encoders for Machine Translationcited by 8
  2. 2023Quantifying the Plausibility of Context Reliance in Neural Machine Translationcited by 6
  3. 2024PORTULAN ExtraGLUE Datasets and Models: Kick-starting a Benchmark for the Neural Processing of Portuguesecited by 5
  4. 2023Impact of Visual Context on Noisy Multimodal NMT: An Empirical Study for English to Indian Languagescited by 2
  5. 2024Towards Chapter-to-Chapter Context-Aware Literary Translation via Large Language Modelscited by 2