# Original or Translated? A Causal Analysis of the Impact of Translationese on Machine Translation Performance

arXiv: https://arxiv.org/abs/2205.02293 · HTML: https://mlsanity.com/paper/2205.02293/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 81 references that are in the map.
- First posted on arXiv 4 May 2022.

## Builds on (most cited first, 23 of 23)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [Improving Neural Machine Translation Models with Monolingual Data](https://mlsanity.com/paper/1511.06709/) (2015)
- [Stanza: A Python Natural Language Processing Toolkit for Many Human Languages](https://mlsanity.com/paper/2003.07082/) (2020)
- [Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation](https://mlsanity.com/paper/2004.09813/) (2020)
- [Facebook FAIR's WMT19 News Translation Task Submission](https://mlsanity.com/paper/1907.06616/) (2019)
- [To Ship or Not to Ship: An Extensive Evaluation of Automatic Metrics for Machine Translation](https://mlsanity.com/paper/2107.10821/) (2021)
- [Revisiting Self-Training for Neural Sequence Generation](https://mlsanity.com/paper/1909.13788/) (2019)
- [BLEU might be Guilty but References are not Innocent](https://mlsanity.com/paper/2004.06063/) (2020)
- [Has Machine Translation Achieved Human Parity? A Case for Document-level Evaluation](https://mlsanity.com/paper/1808.07048/) (2018)
- [Attaining the Unattainable? Reassessing Claims of Human Parity in Neural Machine Translation](https://mlsanity.com/paper/1808.10432/) (2018)
- [The Effect of Translationese in Machine Translation Test Sets](https://mlsanity.com/paper/1906.08069/) (2019)
- [On The Evaluation of Machine Translation Systems Trained With Back-Translation](https://mlsanity.com/paper/1908.05204/) (2019)
- [Translationese in Machine Translation Evaluation](https://mlsanity.com/paper/1906.09833/) (2019)
- [Translationese as a Language in "Multilingual" NMT](https://mlsanity.com/paper/1911.03823/) (2019)
- [IMaT: Unsupervised Text Attribute Transfer via Iterative Matching and Translation](https://mlsanity.com/paper/1901.11333/) (2019)
- [Domain, Translationese and Noise in Synthetic Data for Neural Machine Translation](https://mlsanity.com/paper/1911.03362/) (2019)
- [Using Monolingual Data in Neural Machine Translation: a Systematic Study](https://mlsanity.com/paper/1903.11437/) (2019)
- [Causal Direction of Data Collection Matters: Implications of Causal and Anticausal Learning for NLP](https://mlsanity.com/paper/2110.03618/) (2021)
- [Investigating Backtranslation in Neural Machine Translation](https://mlsanity.com/paper/1804.06189/) (2018)
- [Unsupervised Identification of Translationese](https://mlsanity.com/paper/1609.03205/) (2016)
- [Exploratory Analysis of a Terabyte Scale Web Corpus](https://mlsanity.com/paper/1409.5443/) (2014)

## Built on by (most cited first, 11 of 11)

- [A Causal Framework to Quantify the Robustness of Mathematical Reasoning with Language Models](https://mlsanity.com/paper/2210.12023/) (2022)
- [Quantifying the Dialect Gap and its Correlates Across Languages](https://mlsanity.com/paper/2310.15135/) (2023)
- [Lost in Literalism: How Supervised Training Shapes Translationese in LLMs](https://mlsanity.com/paper/2503.04369/) (2025)
- [Lost in Translationese? Reducing Translation Effect Using Abstract Meaning Representation](https://mlsanity.com/paper/2304.11501/) (2023)
- [Has It All Been Solved? Open NLP Research Questions Not Solved by Large Language Models](https://mlsanity.com/paper/2305.12544/) (2023)
- [Pretraining Language Models Using Translationese](https://mlsanity.com/paper/2403.13638/) (2024)
- [Measuring Spurious Correlation in Classification: 'Clever Hans' in Translationese](https://mlsanity.com/paper/2308.13170/) (2023)
- [Machine Translation Models are Zero-Shot Detectors of Translation Direction](https://mlsanity.com/paper/2401.06769/) (2024)
- [Translation Deserves Better: Analyzing Translation Artifacts in Cross-lingual Visual Question Answering](https://mlsanity.com/paper/2406.02331/) (2024)
- [Mitigating Translationese in Low-resource Languages: The Storyboard Approach](https://mlsanity.com/paper/2407.10152/) (2024)
- [CRAT: A Multi-Agent Framework for Causality-Enhanced Reflective and Retrieval-Augmented Translation with Large Language Models](https://mlsanity.com/paper/2410.21067/) (2024)

## Abstract (arXiv)

Human-translated text displays distinct features from naturally written text in the same language. This phenomena, known as translationese, has been argued to confound the machine translation (MT) evaluation. Yet, we find that existing work on translationese neglects some important factors and the conclusions are mostly correlational but not causal. In this work, we collect CausalMT, a dataset where the MT training data are also labeled with the human translation directions. We inspect two critical factors, the train-test direction match (whether the human translation directions in the training and test sets are aligned), and data-model direction match (whether the model learns in the same direction as the human translation direction in the dataset). We show that these two factors have a large causal effect on the MT performance, in addition to the test-model direction mismatch highlighted by existing work on the impact of translationese. In light of our findings, we provide a set of suggestions for MT training and evaluation. Our code and data are at https://github.com/EdisonNi-hku/CausalMT