# Automatic Classification of Human Translation and Machine Translation: A Study from the Perspective of Lexical Diversity

arXiv: https://arxiv.org/abs/2105.04616 · HTML: https://mlsanity.com/paper/2105.04616/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 36 references that are in the map.
- First posted on arXiv 10 May 2021.

## Builds on (most cited first, 12 of 12)

- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation](https://mlsanity.com/paper/1609.08144/) (2016)
- [COMET: A Neural Framework for MT Evaluation](https://mlsanity.com/paper/2009.09025/) (2020)
- [MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distance](https://mlsanity.com/paper/1909.02622/) (2019)
- [Achieving Human Parity on Automatic Chinese to English News Translation](https://mlsanity.com/paper/1803.05567/) (2018)
- [Has Machine Translation Achieved Human Parity? A Case for Document-level Evaluation](https://mlsanity.com/paper/1808.07048/) (2018)
- [Machine Translationese: Effects of Algorithmic Bias on Linguistic Complexity in Machine Translation](https://mlsanity.com/paper/2102.00287/) (2021)
- [Attaining the Unattainable? Reassessing Claims of Human Parity in Neural Machine Translation](https://mlsanity.com/paper/1808.10432/) (2018)
- [Lost in Translation: Loss and Decay of Linguistic Richness in Machine Translation](https://mlsanity.com/paper/1906.12068/) (2019)
- [Post-editese: an Exacerbated Translationese](https://mlsanity.com/paper/1907.00900/) (2019)
- [Reassessing Claims of Human Parity and Super-Human Performance in Machine Translation at WMT 2019](https://mlsanity.com/paper/2005.05738/) (2020)
- [Unsupervised Identification of Translationese](https://mlsanity.com/paper/1609.03205/) (2016)

## Built on by (most cited first, 4 of 4)

- [A survey of diversity quantification in natural language processing: The why, what, where and how](https://mlsanity.com/paper/2507.20858/) (2025)
- [Automatic Discrimination of Human and Neural Machine Translation in Multilingual Scenarios](https://mlsanity.com/paper/2305.19757/) (2023)
- [The Comparison of Translationese in Machine Translation and Human Transation in terms of Translation Relations](https://mlsanity.com/paper/2404.08661/) (2024)
- [Automatic Machine Translation Detection Using a Surrogate Multilingual Translation Model](https://mlsanity.com/paper/2511.02958/) (2025)

## Abstract (arXiv)

By using a trigram model and fine-tuning a pretrained BERT model for sequence classification, we show that machine translation and human translation can be classified with an accuracy above chance level, which suggests that machine translation and human translation are different in a systematic way. The classification accuracy of machine translation is much higher than of human translation. We show that this may be explained by the difference in lexical diversity between machine translation and human translation. If machine translation has independent patterns from human translation, automatic metrics which measure the deviation of machine translation from human translation may conflate difference with quality. Our experiment with two different types of automatic metrics shows correlation with the result of the classification task. Therefore, we suggest the difference in lexical diversity between machine translation and human translation be given more attention in machine translation evaluation.