mlsanityData to 7 Oct 2026

When Does Translation Require Context? A Data-driven, Multilingual Exploration

arXiv 2109.07446 ↗First posted 15 Sep 2021cs.CL cs.LGcited by 36

Not on the map. Why?

Abstract (arXiv)

Although proper handling of discourse significantly contributes to the quality of machine translation (MT), these improvements are not adequately measured in common translation quality metrics. Recent works in context-aware MT attempt to target a small set of discourse phenomena during evaluation, however not in a fully systematic way. In this paper, we develop the Multilingual Discourse-Aware (MuDA) benchmark, a series of taggers that identify and evaluate model performance on discourse phenomena in any given dataset. The choice of phenomena is inspired by a novel methodology to systematically identify translations requiring context. We confirm the difficulty of previously studied phenomena while uncovering others that were previously unaddressed. We find that common context-aware MT models make only marginal improvements over context-agnostic models, which suggests these models do not handle these ambiguities effectively. We release code and data for 14 language pairs to encourage the MT community to focus on accurately capturing discourse phenomena.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). Builds on 24 of its 51 references that are in the map. First posted on arXiv 15 Sep 2021.

Builds on

24 of 51 references.

References · 24

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2019fairseq: A Fast, Extensible Toolkit for Sequence Modelingcited by 1,486
  4. 2020COMET: A Neural Framework for MT Evaluationcited by 1,024
  5. 2020Stanza: A Python Natural Language Processing Toolkit for Many Human Languagescited by 719
  6. 2018AllenNLP: A Deep Semantic Natural Language Processing Platformcited by 347
  7. 2021Word Alignment by Fine-tuning Embeddings on Parallel Corporacited by 186
  8. 2018When and Why are Pre-trained Word Embeddings Useful for Neural Machine Translation?cited by 125
  9. 2018Context-Aware Neural Machine Translation Learns Anaphora Resolutioncited by 97
  10. 2017Neural Machine Translation with Extended Contextcited by 95
  11. 2018Has Machine Translation Achieved Human Parity? A Case for Document-level Evaluationcited by 88
  12. 2017Evaluating Discourse Phenomena in Neural Machine Translationcited by 87
12 more
  1. 2018Document-Level Neural Machine Translation with Hierarchical Attention Networkscited by 85
  2. 2018Attaining the Unattainable? Reassessing Claims of Human Parity in Neural Machine Translationcited by 58
  3. 2021Measuring and Increasing Context Usage in Context-Aware Machine Translationcited by 48
  4. 2019When and Why is Document-level Context Useful in Neural Machine Translation?cited by 45
  5. 2019compare-mt: A Tool for Holistic Comparison of Language Generation Systemscited by 43
  6. 2017Document Context Neural Machine Translation with Memory Networkscited by 41
  7. 2019Context-Aware Monolingual Repair for Neural Machine Translationcited by 41
  8. 2020It's Easier to Translate out of English than into it: Measuring Neural Translation Difficulty by Cross-Mutual Informationcited by 26
  9. 2019JParaCrawl: A Large Scale Web-Based English-Japanese Parallel Corpuscited by 21
  10. 2021Do Context-Aware Translation Models Pay the Right Attention?cited by 21
  11. 2019Evaluating Pronominal Anaphora in Machine Translation: An Evaluation Measure and a Test Suitecited by 9
  12. 2020Can Your Context-Aware MT System Pass the DiP Benchmark Tests? : Evaluation Benchmarks for Discourse Phenomena in Machine Translationcited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 9; 2024: 13; 2025: 7; 2026: 5.

36 papers.

Sort:

By area:

  1. 2023Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defensecited by 401
  2. 2023Clarify When Necessary: Resolving Ambiguity Through Interaction with LMscited by 93
  3. 2022Interpreting Language Models with Contrastive Explanationscited by 91
  4. 2024Modeling Future Conversation Turns to Teach LLMs to Ask Clarifying Questionscited by 70
  5. 2023Contextual Refinement of Translations: Large Language Models for Sentence and Document-Level Post-Editingcited by 21
  6. 2024Robust Pronoun Fidelity with English LLMs: Are they Reasoning, Repeating, or Just Biased?cited by 17
  7. 2023Identifying Context-Dependent Translations for Evaluation Set Productioncited by 14
  8. 2024Shallow Synthesis of Knowledge in GPT-Generated Texts: A Case Study in Automatic Related Work Compositioncited by 12
  9. 2022A baseline revisited: Pushing the limits of multi-segment models for context-aware translationcited by 11
  10. 2023SLIDE: Reference-free Evaluation for Machine Translation using a Sliding Document Windowcited by 11
  11. 2025What Is That Talk About? A Video-to-Text Summarization Dataset for Scientific Presentationscited by 11
  12. 2024Investigating Length Issues in Document-level Machine Translationcited by 10
24 more
  1. 2024Context-Aware LLM Translation System Using Conversation Summarization and Dialogue Historycited by 7
  2. 2025DocHPLT: A Massively Multilingual Document-Level Translation Datasetcited by 7
  3. 2023Quantifying the Plausibility of Context Reliance in Neural Machine Translationcited by 6
  4. 2024Estimating Privacy Leakage of Augmented Contextual Knowledge in Language Modelscited by 6
  5. 2025HPLT 3.0: Very Large-Scale Multilingual Resources for LLMs and MT. Mono- and Bi-lingual Data, Multilingual Evaluation, and Pre-Trained Modelscited by 6
  6. 2024Is Context Helpful for Chat Translation Evaluation?cited by 5
  7. 2024Reconsidering Sentence-Level Sign Language Translationcited by 4
  8. 2024Analyzing the Attention Heads for Pronoun Disambiguation in Context-aware Machine Translation Modelscited by 4
  9. 2023That was the last straw, we need more: Are Translation Systems Sensitive to Disambiguating Context?cited by 3
  10. 2025You Are What You Train: Effects of Data Composition on Training Context-aware Machine Translation Modelscited by 3
  11. 2023Impact of Visual Context on Noisy Multimodal NMT: An Empirical Study for English to Indian Languagescited by 2
  12. 2024Sequence Shortening for Context-Aware Machine Translationcited by 2
  13. 2024Findings of the WMT 2024 Shared Task on Chat Translationcited by 2
  14. 2024A Context-aware Framework for Translation-mediated Conversationscited by 2
  15. 2023Context-aware Neural Machine Translation for English-Japanese Business Scene Dialoguescited by 1
  16. 2024Using Language Models to Disambiguate Lexical Choices in Translationcited by 1
  17. 2025DOLFIN -- Document-Level Financial test set for Machine Translationcited by 1
  18. 2026Which Tokens Need Context? A Reference-Based Analysis of Translation Responsibility Using Fertility and Entropycited by 1
  19. 2025Unlocking Latent Discourse Translation in LLMs Through Quality-Aware Decodingcited by 0
  20. 2025LLMs for Low-Resource Dialect Translation Using Context-Aware Prompting: A Case Study on Sylheticited by 0
  21. 2026GRUFF: LLM Pronoun Fidelity, Reasoning, and Biases in Germancited by 0
  22. 2026Discourse Dependency: A Continuous Criterion for Translation Difficultycited by 0
  23. 2026The Blindness of Document-Level Translation Evaluationcited by 0
  24. 2026Improving Visual Sensitivity of LLMs on Multimodal Machine Translation with Metric-based Loss Weightingcited by 0