mlsanityData to 14 Oct 2026

Distinguishing Translations by Human, NMT, and ChatGPT: A Linguistic and Statistical Approach

arXiv 2312.10750 ↗First posted 17 Dec 2023cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

The growing popularity of neural machine translation (NMT) and LLMs represented by ChatGPT underscores the need for a deeper understanding of their distinct characteristics and relationships. Such understanding is crucial for language professionals and researchers to make informed decisions and tactful use of these cutting-edge translation technology, but remains underexplored. This study aims to fill this gap by investigating three key questions: (1) the distinguishability of ChatGPT-generated translations from NMT and human translation (HT), (2) the linguistic characteristics of each translation type, and (3) the degree of resemblance between ChatGPT-produced translations and HT or NMT. To achieve these objectives, we employ statistical testing, machine learning algorithms, and multidimensional analysis (MDA) to analyze Spokesperson's Remarks and their translations. After extracting a wide range of linguistic features, supervised classifiers demonstrate high accuracy in distinguishing the three translation types, whereas unsupervised clustering techniques do not yield satisfactory results. Another major finding is that ChatGPT-produced translations exhibit greater similarity with NMT than HT in most MDA dimensions, which is further corroborated by distance computing and visualization. These novel insights shed light on the interrelationships among the three translation types and have implications for the future advancements of NMT and generative AI.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 19 of its 51 references that are in the map. First posted on arXiv 17 Dec 2023.

Builds on

19 of 51 references.

References · 19

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  6. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  7. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  8. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  9. 2023Sparks of Artificial General Intelligence: Early experiments with GPT-4cited by 2,592
  10. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  11. 2020BLEURT: Learning Robust Metrics for Text Generationcited by 1,203
  12. 2023PaLM 2 Technical Reportcited by 1,162
7 more
  1. 2023Large Language Models Are State-of-the-Art Evaluators of Translation Qualitycited by 420
  2. 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 348
  3. 2023Towards Making the Most of ChatGPT for Machine Translationcited by 138
  4. 2023Error Analysis Prompting Enables Human-Like Translation Evaluation in Large Language Modelscited by 87
  5. 2023Large language models effectively leverage document-level context for literary translation, but critical errors persistcited by 80
  6. 2023Unifying the Perspectives of NLP and Software Engineering: A Survey on Language Models for Codecited by 62
  7. 2023Do GPTs Produce Less Literal Translations?cited by 30

Built on by

Not cited yet.