mlsanityData to 7 Oct 2026

The Effect of Translationese in Machine Translation Test Sets

arXiv 1906.08069 ↗First posted 19 Jun 2019cs.CLcited by 49

Not on the map. Why?

Abstract (arXiv)

The effect of translationese has been studied in the field of machine translation (MT), mostly with respect to training data. We study in depth the effect of translationese on test data, using the test sets from the last three editions of WMT's news shared task, containing 17 translation directions. We show evidence that (i) the use of translationese in test sets results in inflated human evaluation scores for MT systems; (ii) in some cases system rankings do change and (iii) the impact translationese has on a translation direction is inversely correlated to the translation quality attainable by state-of-the-art MT systems for that direction.

Key facts. Cited by 49 papers in the map (data through 7 Oct 2026). First posted on arXiv 19 Jun 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 14; 2022: 8; 2023: 8; 2024: 9; 2025: 7; 2026: 3.

49 papers.

Sort:

By area:

  1. 2021The FLORES-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translationcited by 556
  2. 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 344
  3. 2021Experts, Errors, and Context: A Large-Scale Study of Human Evaluation for Machine Translationcited by 333
  4. 2024Tower: An Open Multilingual Large Language Model for Translation-Related Taskscited by 205
  5. 2021To Ship or Not to Ship: An Extensive Evaluation of Automatic Metrics for Machine Translationcited by 161
  6. 2022Crossmodal-3600: A Massively Multilingual Multimodal Evaluation Datasetcited by 100
  7. 2023A Paradigm Shift in Machine Translation: Boosting Translation Performance of Large Language Modelscited by 88
  8. 2021Facebook AI WMT21 News Translation Task Submissioncited by 57
  9. 2022Data Scaling Laws in NMT: The Effect of Noise and Architecturecited by 55
  10. 2021Domain Adaptation and Multi-Domain Adaptation for Neural Machine Translation: A Surveycited by 46
  11. 2021The Dangers of Underclaiming: Reasons for Caution When Reporting How NLP Systems Failcited by 38
  12. 2024Text Embedding Inversion Security for Multilingual Language Modelscited by 26
37 more
  1. 2024URIEL+: Enhancing Linguistic Inclusion and Usability in a Typological and Multilingual Knowledge Basecited by 21
  2. 2024Akal Badi ya Bias: An Exploratory Study of Gender Bias in Hindi Language Technologycited by 17
  3. 2021Language Models are Good Translatorscited by 16
  4. 2021On the Copying Behaviors of Pre-Training for Neural Machine Translationcited by 15
  5. 2021On the Complementarity between Pre-Training and Back-Translation for Neural Machine Translationcited by 14
  6. 2022Understanding Translationese in Cross-Lingual Summarizationcited by 14
  7. 2021Comparing Feature-Engineering and Feature-Learning Approaches for Multilingual Translationese Classificationcited by 13
  8. 2021On the Language Coverage Bias for Neural Machine Translationcited by 12
  9. 2021The USYD-JD Speech Translation System for IWSLT 2021cited by 11
  10. 2022Original or Translated? A Causal Analysis of the Impact of Translationese on Machine Translation Performancecited by 11
  11. 2025Lost in Literalism: How Supervised Training Shapes Translationese in LLMscited by 11
  12. 2023Lost in Translationese? Reducing Translation Effect Using Abstract Meaning Representationcited by 10
  13. 2024An Empirical Study of In-context Learning in LLMs for Machine Translationcited by 9
  14. 2021Beyond Glass-Box Features: Uncertainty Quantification Enhanced Quality Estimation for Neural Machine Translationcited by 5
  15. 2022Bridging the Data Gap between Training and Inference for Unsupervised Neural Machine Translationcited by 5
  16. 2022Explaining Translationese: why are Neural Classifiers Better and what do they Learn?cited by 5
  17. 2023PR-MCS: Perturbation Robust Metric for MultiLingual Image Captioningcited by 5
  18. 2023Breeding Machine Translations: Evolutionary approach to survive and thrive in the world of automated evaluationcited by 5
  19. 2023Measuring Spurious Correlation in Classification: 'Clever Hans' in Translationesecited by 5
  20. 2023Translating away Translationese without Parallel Datacited by 5
  21. 2024Improving Language Models Trained on Translated Data with Continual Pre-Training and Dictionary Learning Analysiscited by 5
  22. 2024On the Evaluation Practices in Multilingual NLP: Can Machine Translation Offer an Alternative to Human Translations?cited by 5
  23. 2021On Systematic Style Differences between Unsupervised and Supervised MT and an Application for High-Resource Machine Translationcited by 4
  24. 2025UPDESH: Synthesizing Grounded Instruction Tuning Data for 13 Indic Languagescited by 4
  25. 2022Towards Debiasing Translation Artifactscited by 3
  26. 2025The Mediomatix Corpus: Parallel Data for Romansh Language Varieties via Comparable Schoolbookscited by 3
  27. 2022Original or Translated? On the Use of Parallel Data for Translation Quality Estimationcited by 2
  28. 2023Automatic Discrimination of Human and Neural Machine Translation in Multilingual Scenarioscited by 2
  29. 2024Translation Deserves Better: Analyzing Translation Artifacts in Cross-lingual Visual Question Answeringcited by 2
  30. 2025EnAnchored-X2X: English-Anchored Optimization for Many-to-Many Translationcited by 2
  31. 2024Multi-perspective Alignment for Increasing Naturalness in Neural Machine Translationcited by 1
  32. 2025Regional Tiny Stories: Using Small Models to Compare Language Learning and Tokenizer Performancecited by 1
  33. 2025Assessing the Political Fairness of Multilingual LLMs: A Case Study based on a 21-way Multiparallel EuroParl Datasetcited by 0
  34. 2025SiamGPT: Quality-First Fine-Tuning for Stable Thai Text Generationcited by 0
  35. 2026Benchmarking Concept-Spilling Across Languages in LLMscited by 0
  36. 2026Cross-Lingual Probing and Community-Grounded Analysis of Gender Bias in Low-Resource Bengalicited by 0
  37. 2026Testing the Deliteralization Hypothesis in Human and Machine Translationcited by 0