mlsanityData to 7 Oct 2026

Has Machine Translation Achieved Human Parity? A Case for Document-level Evaluation

arXiv 1808.07048 ↗First posted 21 Aug 2018cs.CLcited by 88

Not on the map. Why?

Abstract (arXiv)

Recent research suggests that neural machine translation achieves parity with professional human translation on the WMT Chinese--English news translation task. We empirically test this claim with alternative evaluation protocols, contrasting the evaluation of single sentences and entire documents. In a pairwise ranking experiment, human raters assessing adequacy and fluency show a stronger preference for human over machine translation when evaluating documents as compared to isolated sentences. Our findings emphasise the need to shift towards document-level evaluation as machine translation improves to the degree that errors which are hard or impossible to spot at the sentence-level become decisive in discriminating quality of different translation outputs.

Key facts. Cited by 88 papers in the map (data through 7 Oct 2026). First posted on arXiv 21 Aug 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 26; 2022: 15; 2023: 15; 2024: 10; 2025: 13; 2026: 9.

88 papers.

Sort:

By area:

  1. 2021Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translationcited by 333
  2. 2021Systematic Inequalities in Language Technology Performance across the World's Languagescited by 118
  3. 2022A Few Thousand Translations Go a Long Way! Leveraging Pre-trained Models for African News Translationcited by 97
  4. 2021Survey of Low-Resource Machine Translationcited by 92
  5. 2022The Fallacy of AI Functionalitycited by 88
  6. 2021Machine Translationese: Effects of Algorithmic Bias on Linguistic Complexity in Machine Translationcited by 70
  7. 2021BlonDe: An Automatic Evaluation Metric for Document-level Machine Translationcited by 58
  8. 2022Democratizing Neural Machine Translation with OPUS-MTcited by 57
  9. 2022Embarrassingly Easy Document-Level MT Metrics: How to Convert Any Pretrained Metric Into a Document-Level Metriccited by 56
  10. 2021Measuring and Increasing Context Usage in Context-Aware Machine Translationcited by 48
  11. 2021G-Transformer for Document-level Machine Translationcited by 46
  12. 2021The Dangers of Underclaiming: Reasons for Caution When Reporting How NLP Systems Failcited by 38
38 more
  1. 2021When Does Translation Require Context? A Data-driven, Multilingual Explorationcited by 36
  2. 2024A Shocking Amount of the Web is Machine Translated: Insights from Multi-Way Parallelismcited by 33
  3. 2023Escaping the sentence-level paradigm in machine translationcited by 27
  4. 2023Findings of the WMT 2023 Shared Task on Discourse-Level Literary Translation: A Fresh Orb in the Cosmos of LLMscited by 25
  5. 2021Translation Quality Assessment: A Brief Survey on Manual and Automatic Methodscited by 23
  6. 2021Do Context-Aware Translation Models Pay the Right Attention?cited by 21
  7. 2024Fine-Tuned Machine Translation Metrics Struggle in Unseen Domainscited by 21
  8. 2022GigaST: A 10,000-hour Pseudo Speech Translation Corpuscited by 20
  9. 2021The statistical advantage of automatic NLG metrics at the system levelcited by 18
  10. 2021Modeling Bilingual Conversational Characteristics for Neural Chat Translationcited by 18
  11. 2022Learn To Remember: Transformer with Recurrent Memory for Document-Level Machine Translationcited by 16
  12. 2023Document Flattening: Beyond Concatenating Context for Document-Level Neural Machine Translationcited by 15
  13. 2023Identifying Context-Dependent Translations for Evaluation Set Productioncited by 14
  14. 2024Advancing Translation Preference Modeling with RLHF: A Step Towards Cost-Effective Solutioncited by 13
  15. 2021Divide and Rule: Effective Pre-Training for Context-Aware Multi-Encoder Translation Modelscited by 12
  16. 2021Towards Making the Most of Dialogue Characteristics for Neural Chat Translationcited by 12
  17. 2023Document-Level Language Models for Machine Translationcited by 12
  18. 2025Understanding In-Context Machine Translation for Low-Resource Languages: A Case Study on Manchucited by 12
  19. 2021Data Augmentation by Concatenation for Low-Resource Translation: A Mystery and a Solutioncited by 11
  20. 2022Original or Translated? A Causal Analysis of the Impact of Translationese on Machine Translation Performancecited by 11
  21. 2022A baseline revisited: Pushing the limits of multi-segment models for context-aware translationcited by 11
  22. 2025Multilingual Contextualization of Large Language Models for Document-Level Machine Translationcited by 11
  23. 2025Has Machine Translation Evaluation Achieved Human Parity? The Human Reference and the Limits of Progresscited by 10
  24. 2021Contrastive Learning for Context-aware Neural Machine TranslationUsing Coreference Informationcited by 9
  25. 2023A Survey on Zero Pronoun Translationcited by 9
  26. 2021Non-Parametric Online Learning from Human Feedback for Neural Machine Translationcited by 8
  27. 2021Rethinking Self-Supervision Objectives for Generalizable Coherence Modelingcited by 8
  28. 2023Improving Long Context Document-Level Machine Translationcited by 8
  29. 2021Multilingual Document-Level Translation Enables Zero-Shot Transfer From Sentences to Documentscited by 7
  30. 2022Focused Concatenation for Context-Aware Neural Machine Translationcited by 7
  31. 2023Discourse Centric Evaluation of Machine Translation with a Densely Annotated Parallel Corpuscited by 7
  32. 2023Encoding Sentence Position in Context-Aware Neural Machine Translation with Concatenationcited by 6
  33. 2023FRASIMED: a Clinical French Annotated Resource Produced through Crosslingual BERT-Based Annotation Projectioncited by 6
  34. 2023Quantifying the Plausibility of Context Reliance in Neural Machine Translationcited by 6
  35. 2023Challenges in Context-Aware Neural Machine Translationcited by 5
  36. 2023On-the-Fly Fusion of Large Language Models and Machine Translationcited by 5
  37. 2021PAWLS: PDF Annotation With Labels and Structurecited by 4
  38. 2021Towards Personalised and Document-level Machine Translation of Dialoguecited by 4

50 most cited of 88 All 88 (JSON)