# Translating away Translationese without Parallel Data

arXiv: https://arxiv.org/abs/2310.18830 · HTML: https://mlsanity.com/paper/2310.18830/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 43 of its 67 references that are in the map.
- First posted on arXiv 28 Oct 2023.

## Builds on (most cited first, 30 of 43)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Billion-scale similarity search with GPUs](https://mlsanity.com/paper/1702.08734/) (2017)
- [fairseq: A Fast, Extensible Toolkit for Sequence Modeling](https://mlsanity.com/paper/1904.01038/) (2019)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [Beyond English-Centric Multilingual Machine Translation](https://mlsanity.com/paper/2010.11125/) (2020)
- [Word Translation Without Parallel Data](https://mlsanity.com/paper/1710.04087/) (2017)
- [Unsupervised Machine Translation Using Monolingual Corpora Only](https://mlsanity.com/paper/1711.00043/) (2017)
- [Style Transfer from Non-Parallel Text by Cross-Alignment](https://mlsanity.com/paper/1705.09655/) (2017)
- [Facebook FAIR's WMT19 News Translation Task Submission](https://mlsanity.com/paper/1907.06616/) (2019)
- [Unsupervised Neural Machine Translation](https://mlsanity.com/paper/1710.11041/) (2017)
- [Style Transfer in Text: Exploration and Evaluation](https://mlsanity.com/paper/1711.06861/) (2017)
- [Phrase-Based & Neural Unsupervised Machine Translation](https://mlsanity.com/paper/1804.07755/) (2018)
- [Style Transfer Through Back-Translation](https://mlsanity.com/paper/1804.09000/) (2018)
- [Bridging the Gap between Training and Inference for Neural Machine Translation](https://mlsanity.com/paper/1906.02448/) (2019)
- [Translation Artifacts in Cross-lingual Transfer Learning](https://mlsanity.com/paper/2004.04721/) (2020)
- [Style Transformer: Unpaired Text Style Transfer without Disentangled Latent Representation](https://mlsanity.com/paper/1905.05621/) (2019)
- [Margin-based Parallel Corpus Mining with Multilingual Sentence Embeddings](https://mlsanity.com/paper/1811.01136/) (2018)
- [Tagged Back-Translation](https://mlsanity.com/paper/1906.06442/) (2019)
- [Unsupervised Text Style Transfer using Language Models as Discriminators](https://mlsanity.com/paper/1805.11749/) (2018)
- [Attaining the Unattainable? Reassessing Claims of Human Parity in Neural Machine Translation](https://mlsanity.com/paper/1808.10432/) (2018)
- [Unpaired Sentiment-to-Sentiment Translation: A Cycled Reinforcement Learning Approach](https://mlsanity.com/paper/1805.05181/) (2018)
- [Fighting Offensive Language on Social Media with Unsupervised Text Style Transfer](https://mlsanity.com/paper/1805.07685/) (2018)
- [Unsupervised Statistical Machine Translation](https://mlsanity.com/paper/1809.01272/) (2018)
- [The Effect of Translationese in Machine Translation Test Sets](https://mlsanity.com/paper/1906.08069/) (2019)
- [Problems with Cosine as a Measure of Embedding Similarity for High Frequency Words](https://mlsanity.com/paper/2205.05092/) (2022)
- [Scheduled Sampling for Transformers](https://mlsanity.com/paper/1906.07651/) (2019)
- [Reinforcement Learning Based Text Style Transfer without Parallel Training Corpus](https://mlsanity.com/paper/1903.10671/) (2019)

## Built on by (most cited first, 5 of 5)

- [Lost in Literalism: How Supervised Training Shapes Translationese in LLMs](https://mlsanity.com/paper/2503.04369/) (2025)
- [Towards Tailored Recovery of Lexical Diversity in Literary Machine Translation](https://mlsanity.com/paper/2408.17308/) (2024)
- [Multi-perspective Alignment for Increasing Naturalness in Neural Machine Translation](https://mlsanity.com/paper/2412.08473/) (2024)
- [Scaling, Simplification, and Adaptation: Lessons from Pretraining on Machine-Translated Text](https://mlsanity.com/paper/2509.17317/) (2025)
- [An Investigation of Translationese in the Generations of Multilingual Large Language Models](https://mlsanity.com/paper/2608.17399/) (2026)

## Abstract (arXiv)

Translated texts exhibit systematic linguistic differences compared to original texts in the same language, and these differences are referred to as translationese. Translationese has effects on various cross-lingual natural language processing tasks, potentially leading to biased results. In this paper, we explore a novel approach to reduce translationese in translated texts: translation-based style transfer. As there are no parallel human-translated and original data in the same language, we use a self-supervised approach that can learn from comparable (rather than parallel) mono-lingual original and translated data. However, even this self-supervised approach requires some parallel data for validation. We show how we can eliminate the need for parallel validation data by combining the self-supervised loss with an unsupervised loss. This unsupervised loss leverages the original language model loss over the style-transferred output and a semantic similarity loss between the input and style-transferred output. We evaluate our approach in terms of original vs. translationese binary classification in addition to measuring content preservation and target-style fluency. The results show that our approach is able to reduce translationese classifier accuracy to a level of a random classifier after style transfer while adequately preserving the content and fluency in the target original style.